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preface 


It is to be recognized and understood at the outset that this work is 
intended to be an introduction to the field of modern computational 
methods. It is not designed to be, nor does it constitute, a definitive 
treatment of any one or more of the various topics that make up this 
extremely broad area of study. Equally true, however, it is not a super- 
ficial, topical survey of the subject. It is a serious attempt to develop 
from first principles the various computational methods that are basic 
and/or are in general use today, and to indicate the interrelationships 
that exist among them. It also is hoped that the treatment is meaning- 
ful in terms of problem solving, since this was one of our guiding purposes 
throughout its preparation. 

The major parts of the text have been written with second- or third- 
year students of engineering and applied science in mind—students who 
need a general ‘‘overview” of numerical procedures to facilitate upper- 
level course work. It must be noted, however, that certain beginning 
graduate students also have found it meaningful and have obtained 
approval for its inclusion in their master’s degree programs. 

During the past five to six years, over twelve hundred students from 
all branches of engineering, from mathematics, and from physics, at all 
levels from sophomores through first-year graduate students, have been 
“subjected” to this particular presentation. Their enthusiastic response 
to the general approach used has in large measure kept us plodding 
toward eventual publication, even though at times we have been tempted 
to give it last order of priority. Their comments and suggestions for 
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improvement and clarification have been most helpful, and are sincerely 
appreciated. 

Possibly the most troublesome problem with which we were faced was 
our desire to produce a text which could, by the inclusion or exclusion 
of certain chapters and/or sections of chapters, be taken either before 
or after a more “department of mathematics” oriented course in numerical 
analysis. We admit that we have only partially accomplished this. 
Student reaction in this regard, however, has been more favorable. The 
course has generated an interest in the general area of numerical compu- 
tations, and many of the students who have elected to take further courses 
in this area have indicated positively their feelings toward this first 
presentation. Equally true, upper-level students in mathematics who 
have taken this course following a more rigorous mathematical treatment 
have frequently exclaimed that for the first time they ‘‘see’’ how it fits 
together, and how the methods can be used to solve real problem 
situations. 

It is impossible in a text of this kind to present a level of treatment that 
is sufficiently intensive or rigorous enough to satisfy the needs or desires 
of all students who have a serious interest in any one of the subject areas. 
As noted, the primary purpose of this text is to provide an “overview.” 
Therefore, for supplementary but not necessarily secondary reading, 
there is included a selected bibliography of reference texts. 

Finally, the authors wish to express their appreciation to all who have 
directly assisted them in making this work possible: to Charles Fogel, 
Vipperla Venkayya, Jack Blackmon, Andrew Frankus, and others who 
have made many valuable suggestions concerning the order and clarity of 
presentation; to Dwane Moore, Edward Colman, Robie Button, and 
others who checked and rechecked the problems and their solutions; and 
to Mary Basehart, Edna Thill, Lucy Bianchi, and Clara Baideme for 
typing the text. To all we acknowledge our gratitude. 


Rosert L. KETTER 
SHERWOOD P. PRAWEL, JR. 
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1.1 General Engineering Problems 


Problems in engineering can be conveniently classified as 


> c) Steady-state or ;fransient in nature. Typical steady-state 


problems, often referred to as equilibrium problems, are equi- 
librium stresses in structures, steady-state temperature prob- 
lems, steady-state fluid flow, and steady-state flow of direct 
and alternating currents. Electromagnetic energy fields, 
propagation of heat, and propagation of pressure waves in a 
fluid are all typical transient problems. 

In certain cases, because of particular conditions imposed 
by the governing equations, an instantaneous change from 


s 3 ‘ a ee 
one given situation to another may occur. This phenomenon, 


which does not conveniently fit into the steady-state-versus- 
transient classification given above, is referred to as an eigen- 
value or characteristic-value type of problem. Buckling of 
structures, natural frequency of vibration, and resonance in 
acoustics and electric circuits are typical eigenvalue problems. 

Engineering problems also can be categorized according 
to their physical form as discrete-element systems (those 
consisting of a finite number of degrees of freedom) or as 


continuous systems. The steady-state forced vibration of a 


given number of spring-connected masses is the discrete- 
element type. In terms of the amplitudes of the various 
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masses, solution of this problem can be reduced to solution of a set 
of linear simultaneous algebraic equations. Solution of the transient 
problem for this same mass-spring system corresponds to the solution 
of a set of simultaneous ordinary differential equations with prescribed 
initial conditions. 

An example of a steady-state problem in a continuous system is the 
deformation of a plate with prescribed edge conditions and subjected to a 
nonvarying lateral load. Here, the solution is defined by partial differ- 
ential equations with closed boundary conditions (a boundary-value prob- 
lem). Transient heat flow in a three-dimensional medium requires the 
solution of partial differential equations with some open boundary cond1- 
tions (an initial-and-boundary-value problem). 

Certain cases that are observed to be physically continuous may actually 
be solved as if they were distinct discrete elements. The steady-state 
flow of fluid in a pipe network is continuous in physical form; however, 
if the rates of fluid flow in each of the various segments are considered to 
be the unknowns, the resulting equations are—to all intents and purposes 
—those associated with a distinct-element system. Similarly, it may be 
possible in many other cases to group unknown parameters in a manner 
such that the resulting “lumped systems”’ are the discrete-element type. 
Finite-difference mathematical procedures are also used to obtain approxi- 
mations of continuous systems. 


1.2 The Engineering Approach 


While the equations considered by both the engineer and the mathema- 
ticlan may be of the identical type and form, the attitude and approach 
to a problem by the two groups may be markedly different. The solution 
of a differential equation by a mathematician first requires an examination 
of the existence of a solution. Next, properties of the solution are con- 
sidered. i a development of a suitable form for the resulting equa- 
tion(s) is undertaken. The engineer, on the other hand, when faced with 
a physical situation must derive first the governing differential equation(s). 
The actual existence of a solution to the physical problem often is taken 
for granted since the physical problem does exist. \The existence proof 
of a solution to the mathematical model that has been selected to repre- 
sent the physical problem is also often skipped by the engineer. Having 
obtained an answer, within the accuracy required, an engineer should 
then go back and assure himself that the solution satisfies the physical 
problem originally considered. 

Many engineering problems do not lend themselves readily to closed- 


form analytic solutions. In such cases, approximations are required. 
These may take one of several forms: (1) series solutions; (2) direct 
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numerical solution of the differential equations, using whatever computa- 
tional facilities may be available; and (3) solution by analog methods. 

This volume is intended to be an introduction to the general area of 
engineering computations. The presentation has been slanted purposely 
toward application of computational methods to the solution of problems 
in engineering and applied science. 


1.3 Errors 


It should be understood at the outset that most numerical calculations 
are by their very nature inexact. Primarily, the errors are due to one of 
three sources: inaccuracies in the original data, lack of precision in carry- 
ing out elementary operations, and inaccuracies introduced by approxi- 
mate methods of solution. —_——_4 

Of particular significance are the errors due to “roundoff” and the 
inability of carrying, in a given calculation, more than a certain number of 
significant figures. Terms such as “absolute error,” “‘relative error,’ and 
“truncation error’ have a very real meaning, and frequently an analysis 
parallel to one in question must be carried out to establish the reliability 
of a given answer. 

The problem of error analysis due to roundoff was not considered to be 
too difficult when calculations were carried out by hand or by desk cal- 
culators. Added places and/or error terms could be introduced with 
little additional work. However, there is evidence that available ele- 
mentary ‘error theories’? do not seem to be adequate for estimation, 
with any real degree of certainty, of the roundoff and truncation errors 
that result when modern high-speed digital computers are used. As dis- 
cussed in the sections on digital computations, range arithmetic and double 
or multiprecision solutions may be required to afford an estimate of the 
reliability of a given answer. 


1.4 Presentation of Engineering Problems 


No particular form or method of presentation distinguishes an engineering 
problem from any other problem in applied mathematics. Single and 
coupled, ordinary and partial differential equations with initial bound- 
ary values, are encountered, as also are integral equations.| The need to 
sore sete at Tineay and nowlinear simultaneous algebraic equations, to 
optimize with respect to a given condition the accumulative effects of a 
large number of arbitrary variables, and the need to determine the roots 


of transcendental and algebraic equations are also frequently encountered. 
A convenient form for the presentation and discussion of a large num- 
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ber of engineering problems, which is now well established in up-to-date 
undergraduate mathematics study for engineering students, is matrix 
analysis. Because of the importance of this topic, Chap. 4 is devoted 
entirely to its discussion. Extensive use of basic matrix theory is made in 
certain of the methods of solution of simultaneous equations and in 
obtaining the inverses of linear systems. Eigenvalue problems are also 
approached from a matrix point of view. 
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2.1 Introduction: Computers 


Within the past ten years, profound changes have taken place 
in the engineer’s attitude toward, and his methods of approach 
to, the solution of many types of problems. These changes 
have been due primarily to the widespread availability of 
various types of mechanical and electrical computing 
machines. These high-speed devices can be programmed to 
carry out the many routine computational tasks required for 
problem solution, and also to make certain simple decisions. 
This automation of the computational drudgery has allowed 
more time to be devoted to improvements in problem formu- 
lation. It is in this area that most attention is currently 
being focused. 

Three general classes of computers are in use today: the 
digital computer, the analog computer, and the hybrid com- 
puter. As its name implies, the digital computer operates 
with digits or numbers. The analog computer, on the other 
hand, entails use of continuous variables which are related 
to the actual variables of the problem. The hybrid computer 
combines the best of both the digital and the analog computer. 
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2.1.1 Digital Computers 


Most digital computers have the capacity to do only arithmetic. More 
specifically, their mode of computational operation is algebraic addition. 
However, by properly programming the sequence of additions, these 
machines can be made to perform other operations, for example, multi- 
plication and division. Nevertheless, it should be noted that since every 
operation must be reduced to the basic form of addition, digital com- 
puters, by their very nature, are extremely poor “mathematicians.” 
They are, however, very rapid in performing these additions and can 
carry a large number of significant figures, thereby compensating for 
most of their ‘‘mathematical limitations.” 

To solve a particular problem by use of the digital computer, instruc- 
tions concerning the sequence of operations to be carried out must in some 
fashion be transmitted to the machine. In the case of a desk calculator, 
these instructions are usually contained in the operator’s mind. In an 
electronic digital computer, however, the set of instructions must be 
“entered” into the machine, stored, and acted upon with little or no 
interruption by the operator. Since, as pointed out above, this type of 
computer can do only basic arithmetic, all instructions must use only 
basic arithmetic operations. Approximations to rigorous solutions must 
be used when operations other than these are required. Because of these 
approximations and of the other errors that were itemized in Sec. 1.3, 
the accuracy of digital-computer solutions may sometimes be limited. 

It is apparent that normal usage of a digital computer requires of 
the operator a knowledge of the most exacting detail relative to the method 
of solution to his problem. He must also be able to transmit his thoughts 
to the machine. 


2.1.2 Analog Computers 


The analog computer differs from the digital computer in that it makes 
use of continuous variables rather than digits. The variables of the 
computer are in some way related to the variables of the problem to be 
solved. That is, measurements taken of variables on the analog com- 
puter are to some scale proportional to variables in the real system. 
The input and the output for this type of machine are also of a continuous 
nature, representing a real function. Very often the output from an 
analog computer is a graph in which the variable in question is plotted 
as a function of time. 

The analog computer is a very fine ‘‘mathematician.” It does not 
approximate the solution of a given problem numerically as does the 
digital computer. Rather, it solves the given continuous system rigor- 
ously. It should be noted, however, that accuracy on an analog com- 
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puter is usually limited to no more than four significant figures. This 
limitation in accuracy is due to’ deficiencies both in the computing com- 
ponents and in the readout equipment. If this degree of accuracy can be 
tolerated—and it should be noted that in many cases this is better than 
the accuracy of the engineering data—the analog-computer method of 
solution has much to offer. In fact, in some instances, use of the analog 
computer is superior to use of the digital computer. For example, it is 

not necessary to know how to solve a particular problem to be able to_ 
“obtain a solution with an analog computer. The only requirement is that _ 

e problems be correctly formulated. Programming for an analog- 
computer solution can therefore be much less difficult than programming 
the same problem for digital-computer solution. 

Two general types of analog computers are in use today: those of a 
direct analog nature and those of an indirect or mathematical analog 
nature. The direct analog computer is usually a special-purpose device 
made to handle a specific type of problem. The equations which govern 
the action of these analogs are to some scale the same. equations which 
define the action of the prototype. Several examples of this type of 
computer are aircraft or rocket models in wind tunnels, electric field 
plotters, and membrane analogy devices. The indirect or mathematical 
analog makes use of a model—not that of the physical system under 
consideration, but rather of the equations governing that system. Use 
is made of various electrical and/or mechanical analogs of the basic 
mathematical operations: multiplication, addition, integration, etc. The 
indirect analog is a collection of these “simple” or “‘subanalogs,”’ so inter- 
connected as to represent a particular equation. Examples of this type 
are the common slide rule, planimeters, and mechanical and electronic 
differential analyzers. 

Chapter 2 deals specifically with ideas fundamental to the general 
utilization of electronic digital computers. While the examples contained 
in this chapter do not, in themselves, describe any particular “real prob- 
lem,” they describe types of operations used in more complex cases. 
Solutions to specific problem types are contained in later chapters. 

It is assumed that the reader is already familiar with a programming 
language. 


2.2. The Computer Program 


From the above introduction, it is evident that a set of instructions must 
be provided to the digital computer which defines in a suitable form the 
sequence of consecutive operations required to solve a given problem, 
the solution being the reduction of the problem data to a single number 
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or group of numbers which represents the answer. ‘This detailed set of 
instructions is called the computer program. 

Programs for some of the earlier computing machines were marvels of 
complexity when compared with present-day standards. Perhaps the 
best known of these earlier machines was the IBM 650, popular in the 
late 1950s. These computers were coded or programmed in what is 
referred to as machine-language coding. Only a relative few computer 
“experts” could do this well, since it involved representing each arithme- 
tic operation, one step at a time, by a series of digits, some describing the 
operation to be performed and the others indicating where in the com- 
puter these results were to be stored. It was necessary that the program- 
mer be familiar not only with the problem being solved but also with 
the internal workings of the machine. 

Because of this severe restriction, and to make the computer available 
to a larger number of users, there began to appear several problem- 
oriented programming languages. The purpose of these simplified coding 
systems was to bridge the wide gap that existed between the user and the 
computer. Among the more familiar of these languages were Fortran, 
Mad, and Algol. All these had several things in common. All involved 
coding systems stated in normal algebraic notation. All required the 
use of special computer programs referred to as compilers. These com- 
pilers served as translators from the user-oriented algebraic programs to 
their machine-oriented counterparts. Problems coded in one of these 
languages were therefore run in two phases, or passes: first, the compila- 
tion phase; second, the execution phase. 

Progress toward more and more sophisticated programming systems 
has been very rapid over the past several years. This has been made 
possible in large measure through the great strides that have taken place 
in the development of faster and more efficient computer hardware. 
Today, computer systems which can process several programs apparently 
at the same time are becoming common. These latest generation lan- 
guages are sometimes referred to as conversational in that the computer 
appears to be almost human in its response to input. It is to be recog- 
nized, however, that the language used serves no purpose other than a 
means of communication between the user and the machine, and therefore 
can be considered essentially independent of the numerical procedure 
followed in solving a particular problem. 

Problem solution using the digital computer can normally be thought 
of as being composed of the following four fairly well-defined steps: 


WCE RM 7 
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1. Problem formulation 
2. Establishment of method of solution 
3. Coding 


4. Computer executions 
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Note that the first two of these are independent of the particular machine 
to be used, and are problem-oriented. The latter two steps, however, 
depend only on the computer to be used and on its particular coding 
system. Since it has been observed that most of these systems are easily 
self-taught, and since there exist many excellent programmed learning 
texts having to do with this problem, attention will be focused throughout 
the remainder of this text on the development of logical sequences of 
operations for the solutions to given problem types. Concern, therefore, 
will be primarily focused on the “flow” of the computational process. 


2.3 The Flow Diagram 


Exactness, clearness, and conciseness are essential features of a computer 
program. Not only must all contingencies be included, but the basis on 
which a selection is made must also be indicated. 

To minimize the possibility of error and to provide a more desirable 
form for presentation, there has been developed a more or less standard 
form of shorthand for use in digital-computer work. This pictorial 
representation of the logic of a problem solution is called a flow diagram. 

Several conventions have been developed to represent the various 
types of operations needed in a solution. In each, a particular geometric 
form has been selected to represent a specific type of operation. For 
example, in most, a rectangular box or symbol, as shown in Fig. 2.1, is 
used to indicate arithmetic operations. The operation in the case illus- 
trated is the computation of the sum of the product Mz and 5b, followed 
by the substitution of the sum for the variable called A. It must be 
understood that the equal-sign used in a flow diagram for any arithmetic 
operation means that the quantity to the left of the equal-sign is replaced 
by the quantity to the right. That is, any information that might have 
been in A is replaced by the new sum. This being the case, 2 = 72+ 1 
is a perfectly valid arithmetic statement and says that the current value 
of 7 should be replaced by 7 plus 1, or simply incremented by 1. 

The degree of detail included in a flow diagram is entirely up to the 


Fig. 2.2 will accomplish the same objective. The only difference is the 


Figure 2.1 


A=M:-x +b 
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Figure 2.2 


amount of detail that has been included. It should be noted that should 
the last case be used, some other method must be found to indicate the 
method of determining A. 

In addition to arithmetic compute boxes, symbolic representation for 
the other needed types of operations must be defined. For example, 
direction of the flow of data is indicated by an arrow, as shown in Figs. 
2.1 and 2.2. ae 

The operation of decision making is represented by a question bow. 
Contained therein is a question which must be answerable by either a 
“ves” or a “no.” The alternative exits correspond to the two possibil- 
ities. A typical question box is shown in Fig. 2.3. Such a statement 
requires that the computer check to determine whether the fixed-point 
variable N is equal to or less than the number 50. If either is true, com- 
putation proceeds along the path marked A. If AN is greater than 50, 
the answer to the question is ‘‘No,”’ and path B will be followed. 

Notes which are to be included in the flow diagram can be contained in 
circles in the manner shown in Fig. 2.4. 

As shown in Fig. 2.5, input operations are represented by a rectangular 
box with its upper left-hand corner cut off in a fashion similar to that of 
a punched card; output operations are indicated symbolically by a printed 
page. The first box stands for the statement ‘“Take in the value assigned 
to N.” Correspondingly, the second case would read “Put out the cur- 
rent values of the variables z, y, and z, in that order.”’ 


Figure 2.3 
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Figure 2.4 


To illustrate flow diagramming, and to develop a feeling of confidence 
in using relatively simple problems, consider the following examples. 
The chosen problems are not necessarily indicative of engineering-type 
problems. Rather, they have been selected to demonstrate certain 
important features of flow diagramming. It is to be understood that in 
the interest of clarity no attempt has been made to refine the procedures, 
nor to put them in their most concise or ‘‘best’”’ form. 


Example 2.1 It is desired to evaluate the quantity f(z) at x = 2.76, 
where 


f(x) = 22? + 382 +4 Cay) 


Figure 2.6a is a flow diagram that could and normally would be used to 
perform this evaluation. A more detailed flow diagram is given in Fig. 
2.60. 


Example 2.2 Evaluate the sum of the products defined by 
y = » Citi (2.2) 
i=1 
In this expression, ” is a constant which defines the total number of c’s 
and the total number of z’s contained in the problem. The subscript 7 


defines a particular value of c; and z;. Figure 2.7 is one possible flow 
diagram which will perform the summation. 


TN PU 


iS Figure 2.5 
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| f(x) = 2x2 + 3x +4 


(a) 


(b) 
Figure 2.6 


The first step in the flow diagram, i.e., the first step in the computer 
solution to the problem, is to define the value of n, the number of entries 
of each variable. The values for each of the c¢,’s and x,’s then are listed. 
These constitute the input data. 

To provide a correct starting point, it is customary and desirable to 
preset or initialize the dependent variable y. In the case shown, y has 
been set at an initial value of 0. In addition, the “index” or “counter” 
7 has been initialized to the value 1. 
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The first computation carried out by the computer is 
y= 0+ cz (2.3) 


Control then proceeds to the question box, where the current value of i, 
now 1, is compared with the initially given value of n. If 7 = 1 is not 
equal to n, exit will be along the path marked ‘‘No.”’ The index 7 will be 
incremented by 1, and a new value will be computed for y. This second 
computation is 

Vneo = Yprocious 4 C203 (2.4) 
or Y = C121 + Code (2.5) 


Again, 7 is compared with n and, if required, the process is repeated. 
When 7 = n, the current value of y is given as output. The computer 
then stops. 


The repetitive nature of this particular example is characteristic of a 
large number of problem types which effectively utilize the inherent 
capabilities of a digital computer. The process by which computation 
was carried out in Example 2.2 is called looping. That is, the same basic 
computational step was repeated over and over, each time with the substi- 
tution of a new set of numbers. 


Figure 2.7 
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In many cases, it is necessary to loop not just once but twice or even 
several times. Moreover, loops may be contained one within another, in 
which case they are said to be “nested.” ‘To illustrate this concept, 
consider Example 2.8. 


Example 2.3 Evaluate the double sum defined by 


Sum = )) )) (4; + 247B,) (2.6) 
j=l1li=l1 

In this case, both counters 7 and 7 must be controlled. The procedure 
that will be used is as follows: First, set both 7 and J to their initial values 
of 1. Then, holding 7 constant, increment 7 in steps of 1 from its initial 
value to n. A new Sum will be accumulated at each step. When 2 
reaches 7, j will be increased by 1 and 7 reset to unity. The increment 7 
process will be repeated with this new value of 7. When J reaches its 
terminal value m and 7 equals n, the problem will have been completed. 
A flow diagram that will accomplish the desired double summation is 
shown in Fig. 2.8. Note that, asin Example 2.2, it is necessary to provide 
the input data of the problem and to initialize to zero the dependent 

variable Sum. 


Sets of numbers which can be described in the manner of A; and B; of 
Example 2.3 are said to be subscripted numbers. The subscripts are 7 and 
2, respectively. Similarly, c; and x; of Example 2.2 are subscripted num- 
bers, 7 being the subscript. In the application of digital computers it is 
normal procedure to use fixed-point numbers to designate these quantities. 
This is done primarily for the following two reasons: First, a subscript can 
take on only integral values. Second, computations involving fixed- 
point numbers can be carried out more rapidly than those which use 
floating-point numbers. 

A group of numbers the entire set of which can be represented by a 
single-subscripted variable is called a one-dimensional array. The sub- 
script defines the particular entry in question. For example, Aio indi- 
cates the tenth quantity listed in that group known as A. As will be 
shown in Chap. 4, a column matrix is an example of a one-dimensional 
array. 

Certain sets of numbers are listed by the designation of two quantities 
rather than one. When such is the case, a double-subscript notation, 
such as Aj;, can be advantageously used to locate a particular entry. A 
general matrix (see Chap. 4) is an example of just such a two-dimensional 
array. 

A more or less standard notation has been developed for the definition 
of two-dimensional arrays. If we assume that it is possible to arrange the 
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Figure 2.8 


quantities in question in a format of rows and columns, A;; is that quantity 
located in the 7th row and the jth column, as shown in Fig. 2.9. 

In computations involving two-dimensional arrays, nested loops are not 
infrequently observed. Example 2.4 is concerned with the manipulation 
of a square two-dimensional array. 


Figure 2.9 
colin ———— 
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Example 2.4 Given A:;, where both 7 and 7 range in values from 1 
to n, form a new one-dimensional listing called D,;. D, is to be composed 
of the diagonal elements of Ajj. 

The condition defining D; is 


D, = Ai (257) 


The general procedure will be to consider the subscripts 7 and 7 and test 
them for likeness. If, for a particular A;;, the values 7 and 7 are found to 
correspond, that value A;; will be saved as D;. “One possible flow diagram 
that will solve this problem is given in Fig. 2.10. A more concise flow 
diagram to accomplish the same purpose is shown in Fig. 2.11. 

In the program shown in Fig. 2.10, a large part of the effort is devoted 
to scanning the array to establish which particular positions correspond 
toz=y. In Fig. 2.11, on the other hand, it is recognized at the outset 
that the subscripts 2 and 7 uniquely locate each entry A,; in the computer 
storage. Therefore, the proper values can be stipulated and called 
directly from their stored positions without considering the other elements 
in the array. 


In general, the arithmetic parts of a flow diagram are the easiest to 
define. More difficult to specify are those segments which deal with the 
flow of information, branching and looping. For this reason, the remain- 
ing examples in this section contain few arithmetic operations beyond sim- 


Figure 2.10 
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Figure 2.11 


ple substitution such ast = 7+ 1. Attention will be focused on the logic 
of the flow diagram. 


Example 2.5 Consider the problem of constructing a flow diagram 
which will provide as input to a computer a fixed-point number n and n 
floating-point numbers in the form of a one-dimensional array A;. When 
all numbers have been placed in storage, each is to be examined for size 
and then sorted; the largest number being listed first, the second largest 
next, and so on until all have been arranged in descending order of mag- 
nitude. The computer should then give the final arrangement of the n 
numbers as output. 

‘The general procedure that will be used to solve this problem will be 
to select the first given number and compare it with each of the succeed- 
ing numbers. Whenever one is found that is larger than the first, that 
number will be interchanged in position with the first and the process 
continued. The largest number will correspond to the first in the list 
after » — 1 such comparisons have been made. This number will be 
saved as Ay. To determine the second largest number, the same process 
will be repeated, but this time the second number will be used as the 
base and comparisons will be made with the third, the fourth, and so on 
to the nth number. The third and subsequent terms will be established 
in like manner. A flow diagram to accomplish this number sorting is 
shown in Fig. 2.12. 
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Figure 2.12 


In most cases, the best way to examine the working of a flow diagram 
is to use a numerical example. Consider, for example, the following data 
and the flow diagram of Fig. 2.12. 


aan 


(2.8) 
2 


First, the value of n and then the n values of A; are read into the 
machine. It should be noted that in the diagram two indices 7 and 7 
have been used. Index 7 identifies the particular value of A with which 
all the remaining numbers are to be compared. Index 7 identifies the 
remaining numbers. The actual manipulation begins by setting 7 to 1 
and 7 toz + 1, or 2. A check is then made to determine whether A, is 
greater than Ai, that is, whether A; — A: results in a negative quantity. 
Since A; = 7 and A» = 6, it is not the case. Before incrementing 7, it is 
necessary to determine whether all values of A; have been used. Since 
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j = 2 and not n, j is incremented to 3 and A; and Az; are compared. 
A; — Aziscomputed. A, — Azsisnegative. Thus, Ai and A; are inter- 
changed: Ai becomes 8, while A; becomes 7. Sincej = 3 ¥ 4,7 is incre- 
mented to 4, and Ai, or 8, is compared with Ax, or 2. The difference 
A, — Ax is found to be positive; and since 7 = n, the largest value of A; 
must be the first in the list. At this time the group will be arranged as 
shown in Eq. (2.9). 


An 


8 
6 
7 (2.9) 


2 
The next step is to determine whether 7 has become equal to n — 1. 
Since 7 = 1 and n = 4, the answer to this test is “no.” 7 is therefore 
incremented to 2. Correspondingly, 7 becomes 3. The comparison of A3 


and A>» requires that the second and third numbers be interchanged. The 
array now reads 


A 


> ~I Co 


(2.10) 
2 


A continuation of the calculational process will show that Eq. (2.10) is 
the desired array. 


Flow diagrams of the type presented in this text define the logic 
required to solve particular problems. They do not, as was pointed out 
earlier, constitute the actual itemized instruction that must be supplied 
to a particular computer. For completeness, then, and to emphasize this 
point, a Fortran program corresponding to the flow diagram given 
as Fig. 2.12 is shown in Fig. 2.13. The numbers that are circled in the 
flow diagram correspond to the statement numbers in the Fortran 
program. 

_By comparing the flow diagrams of Figs. 2.12 and 2.13, an inconsistency 
will be noted. In the first case, one statement defines the process “‘Inter- 
change A; and A;.”’ Three separate arithmetic statements are shown for 
this same operation in Fig. 2.13. When numbers are interchanged, care 
must be exercised to ensure that none are lost because of destructive 
read-in. Figure 2.13 accomplishes this by first defining a new variable, 
Save, into which one of the variables is placed temporarily. The partial 
flow diagram used is shown in Fig. 2.14. Literally, A; is placed into a 
temporary storage location called Save. Next, A; is moved into location 
A;. (It should be noted that A; in that location is destroyed.) Finally, 
the value that was stored temporarily in Save (that is, A;) is placed in the 
former A; location. The interchange is then complete. 
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DIMENSION A({000) 

400 FORMAT (14) 
101 FORMAT (E14,8) 
READ (5,100)N 
M=N-‘ 
READ(5 A01)(A(I),I =1,N ) 
i= P 
Recs 02 Ic | 
IF(A(I) — A(J)) 6,7, 7 
SAVE = A(J) 
A(J)= A(T) 
A (1)= SAVE 
(ete (O=IN) Seno 
S) dle aes 

GO; 1075: 
8 IF (I-M) 11,10,14 
Gea ae oe 5 

GO TO 4 
10 DO 12 T=1,N 

WRITE (6,101) A(T) 
12 CONTINUE 
14 CALL EXIT 

END 
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Figure 2.14 


Many engineering problems require as part of their solution the scan- 
ning of multidimensional arrays. Consider the following example. 


Example 2.6 Let Ai;, which is a square array of numbers, or a square 
matrix, of n rows and columns be given.t{ The problem is to form a new 
square matrix, the rows of which are the columns of the originally defined 
+ See pp. 40 and 41. 
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A,;, and vice versa. That is, 
A,,(new) = A,;;(original) (2) 


(As will be defined in Chap. 4, the problem is to develop the transpose of a 
given matrix.) 

If we assume that it is desirable to use the same variable A,; for the 
transposed set as for the original set, care must be taken during scanning 
so as not to destroy half the matrix by destructive read-in. This possibil- 
ity is minimized if that part of the matrix which lies above or below the 
main diagonal, which is the diagonal from the upper left or Ai; element to 
the lower right or A,,, element, is considered first and temporarily stored. 
For example, consider 7 ranging from 1 to n — 1 andj fromi + 1 ton. 
(The region in question is shown shaded in Fig. 2.15.) A flow diagram 
that will transpose a square matrix is shown in Fig. 2.16. 

—===— —=té 


The purpose of the latter part of Chap. 2 has been to indicate, primarily 
by example, the basic concepts involved in the construction of flow 
diagrams. As pointed out, the development of these diagrams is a pre- 
requisite to the preparation of the more detailed digital-computer pro- 
grams. In many of the illustrative cases, the problems used have little 
or no direct engineering application. They are included merely to 
demonstrate a particular technique. Certain of the other examples, 
however, will be used extensively in the work to follow. 

It is the opinion of the authors that before flow diagramming of a 
particular method of solution to a given problem can be undertaken, the 
full implications of the proposed method must be understood. Therefore, 
in each of the following chapters where it is applicable, the development 
of flow diagrams is delayed to the last section. Only after the theory has 
been developed, illustrative examples carried out, and problems assigned 
is any computer formulation considered. Moreover, in most cases, only 
one of the methods discussed is chosen for flow diagramming. 


Figure 2.15 
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Figure 2.16 


Prob. 2.1: Construct a flow diagram that will select the smallest of 
n ordered numbers A;, and label that particular number S. 


Prob. 2.2: Construct a flow diagram for the evaluation of n!. 
Prob. 2.3: The average of n numbers is found from the relationship 


n 


Ang = + YA, 


Draw a flow diagram to compute Avg. 

Prob. 2.4: Develop a flow diagram that will put out the absolute 
values of a series of numbers A;. 

Prob. 2.5: The numbers M and @Q are specified. The quantities Q, 
16Q, 149, 16Q, etc., are to be cumulatively subtracted from M. Develop 
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a flow diagram that will determine the smallest number of subtractions 
needed to produce a remainder of less than FE. If a solution has not been 
reached in 100 subtractions, indicate that fact and stop the computation. 


Prob. 2.6: Given two linear functions of x, 
Te jie) Y2 = f2(z) 
develop a flow diagram that will determine the value of x for which 
lyz — y:| < error 


Prob. 2.7: Square roots can be obtained with a digital computer by 
repeated application of the equation 


( * 
Bin = = (a + — 
2 Xi 


where WN = number whose root is desired 
x; = trial value of root 
Liz41 = computed next trial value of root 
Develop a flow diagram in which this process of “iteration” is continued 
until the difference between 2,1: and 2; is equal to or less than a term 
called error. 
Hint: In this case, a possible compute box would be 


1 N 


Prob. 2.8: A quadratic equation can be written in the form 


ax? + br +c=0 


If b? — 4ac is positive, the equation has two real and unequal roots. 
These can be found from the relationship 


—b+ Vb? — 4ac 


ait 2a 


Construct a flow diagram for determining these roots. 

Prob. 2.9: If, in Prob. 2.8, the quantity b? — 4ac is negative, the 
equation has two unequal roots that are complex conjugates. Develop a 
flow diagram for the determination of these roots. 
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3.1 Introduction 


One of the properties of a(square aay has particular 
significance in the solution of a large number of engineering 
problems is its Sa Among other things, determi- 
nants can be used advantageously in solving sets of linear 
homogeneous and nonhomogeneous algebraic equations, in 
determining the nature of the stability of a mechanical or 
electrical system, and in solving many of the problems of 
machine theory. 

In general, a determinant can be considered as a square 
array of numbers and/or symbols, and it is usually enclosed 
by two vertical bars. Two 4 X 4 determinants (or determi- 
nants of order 4) are as follows: 


3 6 2 -1 sina sind sinc sind 

4 1 6 1 a b c d 

0 3 1 2 3 1 0 6 
—2 0 -2 ) 2a 4b c 3d 


Often the letter D is used to represent the value of a 
determinant. 

As indicated in Chap. 2, the individual elements in an 
array are frequently designated by the double-subscript nota- 
tion a,;, Ann X n determinant would be 


24 
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Qi1 Aig 13 Qin 
Qo1 A22 Adz 3 Gon 
D1) G41 G32 Gs3 = * Gan | = las (3.1) 
} 
Ani An2 An3 avs Ann \ 


It should be noted that 7 corresponds to the row and 7 to the column in \ 

which the element in question is located. —— 
Corresponding to every element a;;, there can be written an n — 1 7, 

square array consisting of the original determinant with the 7th row and i) AA: 

the jth column removed. This new determinant of order n —.1 is called | 

the minor of a,; and is designated as M;;. For example, in a determinant 

of order 3, the minor corresponding to a3 is 


Qo1 Ao 
31 39 


My = (32) 


Corresponding to each a;; element, there is also a cofactor, designated by | 


A,;, where 
0) | 


. . Sas 
For example, given the determinant 


3 0 —2 
1 4 1 
3-2 ii) 
4 1 4 1 
Ga — 3 My =|_ | Ay = (Soh a) | 
3 -—2 3. 2 
a32 = —2 305 | t | Ase — (—1)*? 1 i 


3.2. Evaluation of a Determinant 


The method for determining the value of a determinant which is most 
often stressed in high school and-elementary college algebra courses is 
known as expansion by minors or expansion by cofactors. This method, 
developed by Laplace, requires the algebraic summation of the n products 
a;;A;;, where either 7 or j is held fixed while the other takes on the values 
1 to n. 
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Summing along a selected row 2: 
Dz ys aijzAij 
dm (3.4) 


Summing along a selected column 7: 


The cofactor for any element in a determinant of order 2 is the diag- 
onally opposite element with the appropriate sign. . Thus, the value of a 
2 X 2 determinant is 


a a 
Dog =]? | = au (—1)™ azo] + ai2(—1) I @21| 
Q21 22 
or Dox = A11d22 — 412021 (3:5) 


where use has been made of the fact that the value of a determinant of 
order 1 is the value of the element in question. Evaluating the preceding 
example by this method gives the following: 


Expansion about the first column: 


2) 2 
4 1] 0 —2 
Dia mete Leer ee [+ a@(-p | 
a ieee = anne 
+ OCD) 1 ae 


= (3)(20 + 2). — (1)@ — 4) + (8) + 8) = +94 
Expansion about the first row: | 


= in 1 
D = (3)(-1)* Meee 


4 
ms 5 FORD} 5 + (2-9 


= (3)(20 + 2) + 0+ (—2)(—2 — 12) = +94 


It should be noted that the evaluation of an nth-order determinant by 
the method of expansion by minors.requires the evaluation of n determi- 
nants of order n — 1. These, in turn, each require the evaluation of 
n — 1 determinants of order n — 2, etc. The solution of a determinant 
of order 4 would require the evaluation of 


(4) 3 X 3's 
(4) (8) 2 ees 
(4) (8) (2) 1X 1’s 


determinants 27 


or the evaluation of 4! determinants of order 1. The general n X n deter- 
minant would require n! evaluations. From this it should be clear that 
the determination of the value of a determinant by the method of expan- 


sion by minors (or by cofactors) can be extremely cumbersome and time- 
consuming. 


3.3 Special Properties of Determinants 


There are certain special properties of determinants that greatly facilitate 
evaluation. Several of these will be listed here. Their usefulness cannot 
be overemphasized. 


1. A determinant is zero if any two parallel rows (or columns) are 
identical or multiples one of the other. For example, 


SS 


30 6 . 
1 4 214) OD) = 0 
2 1. 4 


2. A determinant is changed in sign if any two parallel rows (or columns) 
are interchanged. For example, expanding about the first row, 


Dad aap 
i Oaevend paeees 
poehit: 29283 
fos Fe 
> ee o it oS 
~s]0 2 4 


3. If the transpose of D is defined to be the determinant D’ whose 
columns are the rows of D and vice versa, the value of D’ is the same 
as that of D. For example, 


} 


oy ives 
D=)|}0 2 4|=-8 

Le, 

oo Vel 
Deed 22 =e 8 

3.4 3 


4. A common term may be factored from all the elements in a line 
(row or column) and placed as a multiple outside the new deter- 
minant. For example, 


23 
0 3| = 2(20) = 40 
‘ny 
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5. The value of a determinant is not changed if a multiple of any one 
of the lines (rows or columns) is added to or subtracted from any 
parallel line. For example, =-2 = 


OR Ae 
2 1s 32 xa eae ve 
=|0 2 42x 2)|=10 2 0|= —-8 
12 3-(2X2);| {1 2 -1 
St 5 a 


3.4 Row or Column Addition a 


Row or column addition, as defined by property 5 above, is particularly 
important in reducing the amount of work required to evaluate a deter- 
minant. Consider the following fourth-order array: 


pie Qo1 G22 G23 Ara (3.6) 


If we assume that it is desired to reduce all elements in the third column 
to zero with the exception of the first element, which is to be reduced to 1, 
a common factor a3 is extracted from row 1 (property 4): 


Qi1 Aye 1 a14 
Giz 13 a13 
D = a43| G21 G22 G23 2a (334) 


Q41 G42 G43 a4 


Row 1 is now multiplied, in turn, by a3, a33, and a43, and the resulting 


rows are subtracted from each of the remaining na , (property 5). 
That is, 


=e aie 1 oe 
13 Q13 Q13 
a1 a12 a4 
Gay — —— Gis. (Gan = —— Org 0) Ga — = as 
D = a3 ie g - (3.8) 
Qi1 a2 Qi4 ; 
Osi Osa Ago, — —— 0339 O05, — —— Gigs 
ai3 Q13 ai3 
Qi1 aio Q14 
day —— Ois 00s — Oa RONG = — Gas 
13 Q13 Q13 


The order of the determinant can now be reduced by 1 by expansion with 
the element 1 in the first row, third column as a “pivot.” 
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Q1 a12 Q14 

Cian maggot 22s aan log on oe 
13 Qi3 ai3 
ai 12 a4 

D = 43 | @31 — — Oley OEY =" 8s (01 FoI (3.9) 

Q13 Q13 Q13 

* ai #s 2 Qi2 a4 q 

Ciel CLAS 42 — —— Gag sg — —— Oe) 

Q13 a13 A138 5p) 


By repeated application of this process, any determinant can be reduced 
to second order, which is evaluated easily. Consider the following 
example. The line enclosed by dashes is to have all elements reduced to 
zero save one, the pivotal element, whereas appropriate multiples of the 
lines underlined by dashes are to be subtracted from parallel lines. 


Sates 119 aT 
to ee ae ae “7 
oe Te ol ge | kee aa he) 
leo 12 Dn Ae ew 


3.5 Pivotal Condensation 


A more formal method for the evaluation of a determinant, which can be 
derived from the above and which is ideally suited for digital-computer 
programming, is that of pivotal condensation. With the given determinant 


ii Omi bE 
Qo1 Qo2 * °° Ge 

1D) = y (3.10) 
An1 Ane cate Ann 


and the assumption that the 1-1 position is to be the pivot point, the order 
of the determinant can be reduced by 1 by using the following: 


al = (2 G11 12 a1 13 .. . | O11 Ain 
Sa Go1 22 G21 A22 G21 Gon 
il G11 Ay G11 «13 es ei (3.11) 
oe = 31 a Q31 a : 
(a41)"~? G31 32 31 33 31 3n 
ai1 Aie2 G11 13 Sy: Qi1 in 
Ani OAn2 Ani An3 Ani Onn 


Successive application of this general formulation can be used to reduce 
the order of the determinant to a 2 X 2, which is solved readily. 
The example in Sec. 3.4 could be evaluated by the method of pivotal 
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condensation as follows: 


aa 
1 0 3 
aah oe 1 zi | 12 
we On| OV ia 25 0 : At) S03 eG 4 45 
TPO 5 dee 


One of the conditions necessary to obtain a solution by the method of 
pivotal condensation is that the value of the element ai: must be other 
than zero. If this this 1 is not the case, a rearrangement of lines is necessary 


by factoring a, out of row 1 or anni 1 prior to reduction, that is, = 
next-lower-order determinant will then be the ‘‘old’”’ value of ai; divided 
by 1.0 raised to the (n — 2)nd power, or simply au. 

If certain terms in the array are very large in comparison with those 
remaining, and if capacity of the computer to be used is relatively small, 
it may be desirable to factor the largest element from either its row or 


its column. 
4 : : a b Cc 
Prob. 3.1: Given the relationship ——— = — = —— evaluate the 
sin A snB sind 
determinant 


Prob. 3.2: Evaluate by expansion by minors, row or column addition, 
and pivotal condensation the following determinants: 


Ona on ee (Oye lie Pes ae 
= Sf 01 ifs) Gus10 
= Ou eee ae) 1,4 10 20 
ee cif a ie agen, ee 

(eo) 2 0 0 (d)|A -B C (e) 0 —C12 +C13 

12 0) dx dy 0 +1. 0 — Co3 

ee O11. Oat 0 dx dy —¢13 +e; ~—0 

WO gh @ 


Prob. 3.3: By any of the methods discussed, solve the following 
determinant and arrange the resulting polynomial equation in order of 
descending powers of e. 
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Gilig =2 €12 613 
€12 GEpY SK3 €23 
€13 €23 GS = G 


Prob. 3.4: Verify, using a general nth-order determinant and ay as a 
pivot point, that row or column addition and pivotal condensation result 
in the same n — 1 determinant. 

Prob. 3.5: Determine the pivotal-condensation reduction equation for 
expansion of a general nth-order determinant about position a3. 

Prob. 3.6: Verify that the value of a determinant having zeros in all 
elements to the left (or to the right) of the leading diagonal (i.e., the aii 
elements) is equal to the product of the diagonal elements. 

Prob. 3.7: Determine the positive value of m such that the following 
determinant is zero: 

ae 2 
2 iL = i 


3.6 Flow Diagram for Pivotal Condensation 


In this section a flow diagram will be developed for the evaluation of an 


nth-order determi by using pivotal condensation. With the excep- 
tion of the added normalization step (Step III in Fig. 3.1), the equation 
governing the solution is that given previously as Eq. (8.11). 

The sequence of presentation that will be used in this and the remaining 
flow-diagramming problems is first to define in very broad terms the major 
steps that constitute the solution to the problem in question. This gen- 
eral solution then will be followed by detailed discussions of the opera- 
tions necessary to accomplish each major step. Finally, the entire flow 
diagrams will be given. 

Figure 3.1 defines the major operations to be carried out when evaluat- 
ing a determinant by pivotal condensation. For reference, each is desig- 
nated as a step in the solution. 


Sr 

Figure 3.2 is a flow diagram that will carry out Step I of Fig. 3.1, the 
entering of data and initializing of quantities. The notes in Fig. 3.2 
have the following significance: 


1. n is the order of the determinant to be evaluated. 

2. n* elements a;; are read into the computer. 

3. A special counter k is initialized to 1. Its purpose is to keep track 
of the number of “pivots,” or, alternatively stated, the number of 
reductions by order 1 of the original nth-order determinant. On 
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Bring in all data and initialize any special counters. ial 


| 


Check to see if a,, =O. 
If it does, check Gp, , etc. Continue until an a;,4 is 
found which does not equal zero. Interchange that row 
with the first. (If all a;4's are zero, the value of the 
determinant is zero.) 
2 i 
Normalize with respect to the new @44. 


Save this value g,, and determine the elements of a 
new 7-1 determinant, using the -a,, position as a pivot. 


Replace the elements of the old determinant by the 
computed new values and check to see if the new 
determinant has been reduced to the first degree. If 
not, repeat each of the arithmetic steps listed above. 


Multiply all saved values of a,, to obtain the value of 
the determinant . 


Figure 3.1 


Notes 


Step I 6 
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completion of the problem, it should have a value equal to n — 1. 

4. nn is a dummy variable set aside to retain the original order of the 
determinant. 

5. Adj is a variable used to compute changes in sign that are introduced 
into the solution if there is a rearrangement of rows. 

6. This is the starting point for Step II of the solution. 


Strep II 


Check to see if a1. = 0. If it does, check a, etc. Continue until an 
a;; 1s found: which does not equal zero. Interchange that row with the 
first. (If all ai’s are zero, the value of the determinant is zero.) With 


Figure 3.3 


Notes 


_—_—> 7 { 


o% 


{5 
# Sg Bee: mee, F 
FoisT Ponte 
ans a 
(2 Noe 2 
Yes 
Cs 
ey 
MV o> 
QY an 
x 
3 
5 : 
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Step I 
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regard to the notes shown in Fig. 3.3: 


1. The values of ‘7 and 7 are both equated to 1, as the first ai; to be 
considered is a11. The number 100 is a point of entry (for looping) 
to be used when considering the entire diagram. 

2. ay, is checked first to determine whether it equals zero. If the check 
is positive, 7 is incremented by 1 and az: is examined. This process 
continues until either 7 becomes equal to n, at which time the pro- 
gram stops, or a value a, is found which is not equal to zero. 

3. If ai, is not equal to zero, the “No” exit of note 2 is followed, and 
the answer to question 3 is ‘“‘yes,’’ with control passing to point 2. 
If, however, a1. = 0 and an a, ¥ 0 is found, row 1 must be inter- 
changed with that particular 7th row. 

4. The general procedure for interchanging rows was discussed in 
Example 2.5. A flow diagram was given as Fig. 2.14. 

5. If a rearrangement of rows is required, the value of the determinant 
must be multiplied by the quantity (—1). (See property 2 in Sec. 
3.3.) Since Adj was initially set equal to 1, the first such rearrange- 
ment causes Adj to equal —1. Additional switching, at later stages 
in the solution, will result in other changes in the sign of Adj. 

6. Control passes from this point to Step III. 


Strep IIT 
Normalize with respect to the new ai:._ Save this value aj; and determine 
the elements of anew n — 1 determinant, using the aj; position as a pivot. 
This part of the flow diagram is shown as Fig. 3.4, with notes as follows: 


1. At this point, the current value of a3, is placed in Save,. (If k = 1, 
and no row rearrangement was made, Save, will contain the original 
a1.) Each reduction by order 1 of the original nth-order deter- 
minant produces a new ai; element, and thus a new pivot, each of 
which must be saved. The value of k defines the number of reduc- 
tions of order 1 that have been made and thus indicates the stage 
at which a particular ay; was calculated. 


2. Prior to this stage, 7 was equal ton. Since a; is to be divided by ay; 
(that is, quantities are to be normalized with respect to a11), 7 must 
be reset to 1. It should be noted that this gives a 1.0 for the pivot 
value at all times. 


3. Again, the values of 7 andj are reset to 1. By using the 1-1 position 
as a pivot, a new determinant of order n — 1, defined as Anewi3;, is 
calculated. 


4. candj at this stage are both equal ton — 1. Control passes through 
point 3 to Step IV. 
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\e Figure 3.4 
H, Ai T+) 


Or) | ) CiGe a+) 
Step IV 
Replace the elements of the old determinant by the computed new values 
and check to see if the new determinant has been reduced to the first 
degree. If not, repeat each of the arithmetic steps listed above. 
In Fig. 3.5, the following are noted: 


1. Beginning with 7 and 7 equal to 1, the previous values of ai; are 
replaced, one at a time, by Anew,;._ The resulting determinant is 
one order less than the case just preceding. 

2. At this stage, a test is required to determine whether n has been 
reduced to 2. If it has not, n is further reduced by 1 and control 
is sent back to point 100. The entire process then is repeated. If, 
however, 7 is equal to 2, the current determinant is of order n — 1, 
ay, is the only element remaining, and control passes to Step V. 
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4 Ate 
(os 


Notes 


Step IV 
Figure 3.5 
Figure 3.6 
Notes 
i 
[ Prod = Oy Ady ‘| 
Prod = Prod - Save, | 2 
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Stmp V 
Multiply all saved values of a1: to obtain the value of the determinant. 
In Fig. 3.6, the following are noted: 


1. The value of the order of the original determinant is retrieved from 
its stored location nn. The current value of a1; is multiplied by the 
value Adj and, depending on the number of row interchanges that 
have been made during the course of the solution, it can have either 
a plus or a minus sign. 


2. All the previous ai:’s are taken from storage (Save,) and in turn 
multiplied by the product of ai: Adj. The final value of the 
determinant is obtained after n — 1 ai:’s have been incorporated 
into the product, the nth a1; having been used already in the initial 
product ai, ° Adj. 


Figure 3.7 (pp. 38 and 39) is a complete flow diagram for evaluating 
an nth-order determinant by pivotal condensation. Again, it should be 
understood that this 1s not a computer program. It is a diagram, indepen- 
dent of computer make or size, which describes the logic of the solution and 
indicates the flow of information. It is a necessary step in the writing of 
a detailed computer program, but it is not the program itself. 
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n 


<P> : 


Adj= Adj-1) | _ 


J=4 


To 
a7; = a4; + Saveg 


Figure 3.7 Flow diagram for pivotal condensation. 
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To 
f= 


47 = O47 + Save, 
ae 
/=4 


J=4 


Anew s7= 0744 + 44-7 44 Tear 
ae: 
0 


/=/+4 


Prod=ay, -Ad/ 


Figure 3.7 (Continued) 
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4.1 Introduction 


As was pointed out in Chap. 1, matrix theory offers a con- 
venient means of defining and solving many engineering 


problems. 
To illustrate what is meant by a matrix, suppose that m 
quantities y1, Y2, . . . , Ym are related to n quantities 21, Xs, 


. » &n by the relationship 


= y Oj; @=— 1,2) 5 =>) (4.1) 


gi 


In expanded form, these equations are 


Yi = Qiit1 4 Ojys%. + 9 = a Ginta 
Yo = Gait + Goxte + * * * + Goats (4.2) 
Ym = Amit, —- Amete + > * > + Gant 


By defining a certain ordered way of reading, these also can 
be written in the following somewhat different form: 


40 
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Y1 aii ie Ain 11 
Ye A21 Are Aon || Le 
a tear BS Ste eae | (4.3) 
Um Qm1 QAm2 °° * QAmn Xn 
If the column of m entries y:, y2, . . . , Ym is denoted by the single letter 
[Y], the column of n quantities x1, x2, . . . , tn by the letter [X], and the 
m times n quantities a,; by the letter [A], that is, 
Yi v1 G11 Ai2 Qin 
Y2 v2 G21 Are Aen 
'Y] = Ix] = ie eae 
Ym In Qmi QOm2 °° * OAmn 
the original set of equations can be written equally well as 
[Y] = [A][X] (4.4) 


The m quantities y; (that is, [Y]) represent an m-row one-column array 
of numbers or symbols. Then quantities z; similarly correspond to an 
m-row one-column group.. By-definition, a rectangular array of numbers 
or symbols consisting of one column is called a column matrix or column 
vector, and the individual entries are the components or coordinates of 
the vector. Correspondingly, an array consisting of a single row is called 
a row vector. The array of coefficients a;; (that is, [A]) is called a coeffi- 
cient matrix or just a matrix. In different forms, 


AN ae a ste 2 Ree 7 tl) 2,2. nn) (4:5) 


A matrix, then, is nothing more than a rectangular array of numbers and/or 
symbols with each element being distinct and separate. 

By comparing the expanded form of Eqs. (4.2) with the matrix formu- 
lation of Eq. (4.3), it is seen that y; is found by applying the 7th row of 
[A] to the column vector [X] and summing all the products thus obtained. 
That is, 

Ys = Anty + Ate + -- * + Gindn (4.6) 


It will be shown later that this process of row-to-column multiplication 
and summation is actually the defined process called matrix multiplication. 

Matrix formulation has the advantage of brevity. This, however, 
may not be its greatest virtue. Given the set of equations 


[Y] = [A][X] 
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Figure 4.1 


or its alternative form 
yi = Yair (is= 6125 Seana) 


it is evident that from a mathematical point of view the particular column 
vectors [X] and [Y] (or 2; and y;) are not in themselves important. The 
quantity of significance is the coefficient matrix [A]. It is an abstract 
BRST of a linear transformation that exists relating the x’s and 
the y’s. 
_Consider t ing ormation from a rectangular 
Pn A oa ae uv system having the same 


origin (see Fig. 4.1). The governing equations are 


u=xcosatysina 


(4.7) 


v= —zxsna+ycosa 


In matrix notation these equations are written as 
| 7 | cosa sin ;| H (4.8) 
v —sina cosa||y 
u x 
or ee = [A] * | (4.9) 


where [A] is the matrix of the coefficients of x and y and is square (equal 
number of rows and columns). The elements u and v and z and y con- 
stitute column vectors. Assume that it is desired now to effect a similar 
transformation from wo to a still different system rs by rotation through an 
additional angle 8. The corresponding matrix equation is 


| = 1B) [4 (4.10) 


where (B= [- ae pal ; (4.11) 
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In matrix form, it is easy to anticipate the result of a multiple trans- 
formation from zy to wv and then to rs. 


K pas | = [BIA] 7 | (4.12) 


It is equally easy to describe a similar multiple transformation in an 
n-dimensional space. 


Ty U1 U1 
To U2 v2 

=[B]| | = [BI[A)| _ (4.13) 
Tn Un Ln 


(It should be noted that in this case the coordinates have been designated 
Aer nti, Wey eG UDG i I, . + . 5 72 rather than as 
different letters for each of the 3n coordinate directions.) 


4.2. Special Matrices 


In the work which follows, use will be made of certain special matrices, 
which are summarized here for definition and future reference. 


4.2.1 The Square Matrix 


A matrix having an equal number of rows and columns is called a square 


matrix. 
Ak DETERMINATE 
Qi iz Qin 
on oe ws. > Le sae a [ais] (2 y L 2; : ) dS 1, 2, , n) 
Ani ano Ann 
(4.14 


4.2.2 The Diagonal Matrix 


A square matrix in which all elements not on the leading diagonal (that 
is, the 77 position) are zero is called a diagonal matrix. 

an 0) Goo 0) 

Ue SC ae (4.15) 


py) exioileiel ose, (el, cat retyyte Ney forse, 
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4.2.3. The Unit or Identity Matrix 


A diagonal matrix in which all the diagonal elements equal 1.0 (that is, 
a; = 1) is called a unit matrix or an identity matrix. It usually is 
designated by the letter [J]. 


1 O 0 
7] en oi (4.16) 
0 O 1 


4.2.4 The Upper Triangular Matrix 


A matrix in which zeros exist in all elements to the left of the 7 position 
is called an upper triangular matrix. It frequently is designated by the 
letter [U]. 


Ui SUyo esas Wins 
[ice |e lies Be (4.17) 
0 0 Te 


4.2.5 The Upper Unit Triangular Matrix 


An upper triangular matrix having all elements in the leading-diagonal 
positions equal to 1.0 (that is, a;; = 1) is called an upper unit triangular 
matrix. It frequently is designated by the letter [7], and the individual 
entries by &,;. 


1 tie bs 
0 1 7 

dia eeteern =e (4.18) 
0 O il 


4.2.6 The Lower Triangular Matrix 


A matrix in which zeros exist in all elements to the right of the 27 position 
is called a lower triangular matrix. It frequently is denoted by the letter 
[LZ] and the entries by ¢,;. 


fy 0 
& ly 0 

[Eis ' | is Si ae vega (4.19) 
Can £n2 tan 


4.2.7 The Nullor Zero Matrix 


A matrix in which zeros exist in all elements-is called a null or zero 
matrix. It is usually designated by the symbol [0] or just 0. 
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4.2.8 The Transpose of a Matrix 


The transpose of an m X n matrix [A] is ann X m matrix whose columns 
and rows have been interchanged. It is usually denoted by a prime 
(that is, [A’]). 


Qi1 Aye Qin Qi1 Aa Qml 
Qo Q29° 52 | Go Qi2 Go2 * * * Ame 

Pala Meal 7h i Pe Ss eo erties as es, "| (4.20) 
QAm1 An2 Amn Qin Qeon Amn 


4.3 Elementary Matrix Operations 


Since by definition a matrix is a rectangular array of numbers, each of 
which is distinct and separate, two matrices are equal if and only if all 
their corresponding elements are equal. That is, 


[Are] (4.21) 


if aij = bi; (CBM nee aS es ae ee 


4.3.1 Matrix Addition 


A matrix [C] is the sum of matrices [A] and [B] if each of the elements in 
[C] equals the sum of each of the corresponding elements in [A] and [B]. 
That is, 


Cig = Ai == bi; (4.22) 
Consider the following equation: 
»: AijXj — » bij; = 0 (a a 1, 2; 6 Gp i?) (4.23) 
j= gal 
or Gp bs)te= 0 GH 1,25... 59) (4.24) 
j=1 
or, in matrix forms, 
[AX] — [Bix] = [lA] — [Bie] = [0] (4.25) 
or 
G11 + Ai2 Qin 1 bir Dae bin x1 
G21 22 Gon L2 bor bee bon Hi) 
5 ash Ob ceeteancbeecmenr 04h 0 ase egee Ae : 0) (4.26) 


Qni Qn2 °° * Ann Ln Bri Dre te Une Xn 
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Matrix addition requires that 


[C][X] = [A — B][X] = [0] (4.27) 
In expanded form, this is 
Qu — bir Gig — die Qin — O1n || 21 
Qo1 — bor dee — bee Aon re te | _ [0] (4.28) 
Ani — On1 Ang — One Ann — Onn} L&n 


4.3.2 Matrix Multiplication 


To illustrate the general concept of matrix multiplication, consider the 
following set of equatiops in 2;: 
1121 + A12%2 + Ai3%3 = fi 
or [A]_X] = [F] (4.29) 
1X1 + Ag2%2 + Ao3t3 = fo 


Assume, now, that x1, x2, and x3 are defined in terms of two other vari- 
ables yi and yp. 


“1 = biry1 se DisY2 
Io bey se booYe2 that is, [B][Y] = [X] (4.30) 
3 = baiyi + dsoYye 


It therefore would be possible to write the original equations (4.29) in 
terms of the two variables y; and ye. 


C1141 + Cro = fi 
or ~AIC]EY] 


[F] (4.31) 
Co1Yi + Co2Y2 = fo 


By substituting from Eqs. (4.30) the values of the x’s into Eqs. (4.29), the 
matrix corresponding to [C] of Eqs. (4.31) is 
bir + Qyobe1 + aigd bia + Ayobo2 + aygd 
C ae 411011 12421 13431 o 12 12422 13432 
C1 jena + Ge2bo1 + Ao3b31 a + Ar2b22 + Gosbs2 a2 


As suggested earlier in this section, the total process could be defined 
equally well as 


[A][B] = [C] (4.33) 
or 


bi bie 
Qi1 Ging 13 ‘s 

91 boo 
Qo1 A222 Ads b 

31 O39 


= Ee + Giebe + 13631 A11bi2 + A12be9 Se 13 bse 4.34 
Qoibiy + Geebo1 + Qo3b31 eid12 + Q22b22 + 3 bse ( i ) 
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It should be noted that matrix multiplication consists in row-to-column mul- 
tiplication and summation. For example, the 1-1 position in matrix [C] is 
obtained by multiplying and summing the elements of row 1 of matrix 
[A] to those of column 1 of matrix [B]. Similarly, ci. is found by multi- 
plying and summing row 1 of matrix [A] with column 2 of matrix [B]. 
Having considered the special case of multiplying a 2 X 3 matrix by a 
3 X 2 matrix, consider now the general statement of the matrix multi- 
plication problem. Assume that the following two transformations are 
given: 
Ar [Y] = [A][X] and [X] = [BI[Z] (4.35) 


In summation form, these are 


i — (4.36) 


iM 
& 
S. 
& 
ee. 

~ 

l| 
oe 
ko 

= 
>» 4 


&. 
_ 


| 
Mis | 
= 

> 

rR 

KS 


and Pe (Gazz ASO 28. 3) (4.37) 


ta 
ll 
a 


y: in terms of z is then 


q 
ay d bine A ») y ai; Din2e (4.38) 


k= 171 


ina 


Since it is the composite effect of the two transformations that is being 
sought, 
[Y] = [C][Z] (4.39) 


g 
or Yi = » Cik&k (@ = |, 2, 56 5 a) (4.40) 
k=1 
A comparison of Eqs. (4.38) and (4.40) yields 
cn = >, aigdyp (creme 2 Meme Sei ves Lar Nite Lig) (4.41) 
1 


Matrix multiplication [C] = [A][B] requires that the elements of row 7 
and column k of the matrix [C] equal the sum of the pairs of products of 
the elements of row 7 of [A] and column k of [B]. 

It should be noted that [Y] = [A][X], where LX] is a column vector, is 
a true matrix multiplication. 


4.4 Special Considerations Concerning 
Matrix Multiplication 


From the development of Eqs. (4.41) it is evident that the matrix multi- 
plication [C] = [A][B] has meaning only when the number of columns of 
[A] equals the number of rows of [B]. 
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4.4.1 Multiplication by the Identity Matrix 


The multiplication of any matrix by an identity matrix yields the original 
matrix. 


[A] = [Z][A] = [A][Z] (4.42) 
or 
ay1 A192 Qin 1 O 0 ai1 12 Qin 
Qo1 Are on Ort 0 - G21 Ao2 Aon 
Qm1 Am2 Amn 0 0 1 Am1 Am2 Amn 
(4.43) 


4.4.2 Matrix Multiplication Is Associative 
[A][B C] = {A BI[C] (4.44) 


If [A] = a, [B] = by, and [C] = cx, the zk position of the product 
[A][B] = [A B] = [D] is [from Eqs. (4.41)] 


dis = 2, abs (4.45) 
and the z¢ position of [A B][C] is 
2 dinCee = my s AsjdjuCre (4.46) 
Similarly, the product [A][B C] = [A][E] is (for the 7é position) 
» Aije;¢ = ») y AisdjnCee (4.47) 
j j 


All that differs in Eqs. (4.46) and (4.47) is the order of summation. 
Matrix multiplication therefore is associative. 


4.4.3 Matrix Multiplication Is Not in 
General Commutative ([A B] ¥ [B A]) 


To illustrate this condition, consider the multiplication of two 2 X 2 
matrices [A] and [B]. 


bir + Gizbo1  Airbi2 + diab 
A B) = 11911 12021 11012 12022 
[ ] Be aia Aooba1 Agibie + | (4.48) 
j bi1@i1 + by0001 birdie + Diode 
and BAe : 
es + boeGe1 beidi2 + booQeo ee) 


From Eqs. (4.48) and (4.49), it is evident that only for certain special 
values of the a,;’s and 6,;’s will the two be equal. 
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4.4.4 Matrix Multiplication Is Distributive 
[A][B + €} = [A][B] + [A][C] (4.50) 


(Note: matrix addition.) If [A] = aj, [B] = bj, and [C] = cx, then from 
Eq. (4.23) 


[B + C] = by + ce (4.51) 
The entry in the 7k position for the product [A][B + C] is 
> a:;(b;, + cy) = » Us 0s 42 » OijCjx (4.52) 
J j j 


From Eq. (4.21) it is also evident that Eq. (4.52) also represents the 
matrix sum of the two multiplications 


[A][B] + [A]IC] (4.53) 


The distributive condition therefore is demonstrated. 


4.4.5 Multiplication by a Constant 


The multiplication of any matrix by a constant (scalar) results in a new 
matrix, every entry of which is multiplied by the constant. 


ERT, ORT AIAG ACO as Oe 5) Qi1 Gig * * * Gin 
e Qe1 22 Gen O- a 0 Qe1 Are Gon 
Qn1 Ane Ann 0 0 a Qni Ane Ann 
Qt” AQjoy 2° = Adi, 
ade ade2 ee Adon (4 54) 
Adni AAne AAnn 


4.5 Partitioning of Matrices 


It is evident that as matrices become larger, the amount of work required 
to carry out a given operation (e.g., multiplication) may increase at a 
fast rate. Computer storage capacity may also become a problem in solv- 
ing the larger problems. To facilitate solution, it is often desirable to 
partition matrices into subgroups or submatrices and to operate on these 
smaller units. 

Consider the 4 X 4 matrix 


[A] zl Goi G22 G23 | A24 (4.55) 
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For the divisions indicated by the dotted lines, there results a matrix of 
order 3, a 1 X 3 row vector, a 3 X 1 column vector, and a number das. 
This can therefore be written as shown in Eq. (4.56). 


14 
[4], sek eee (4.56) 
dani oan (ae 
See ns Sires 
- Tae Hae ss | | (4.57) 


where the column vector (@14,@24,@34) is denoted by [H3,1] and the row 
vector (@41,442,d43) by [Giz]. This type of partitioning, known as border- 
ing, has particular advantage in the solution of certain types of engineer- 
ing problems. 

A special array, which is “naturally” partitional and which is fre- 
quently observed, consists of square subsets lying along the leading diag- 
onal with all other terms equal to zero. 


oo Oana on [= [42l P] casa) 


where 
433 A341 35 
and [Asoo] = | G43 Gag 45 (4.59) 
G53 Asa 55 
To illustrate multiplication on a partitioned matrix, consider the 
following: 


[A][B] = [C] (4.60) 


where [A] is a 2 X 3 and [B] is a 3 X 2 matrix. From Eq. (4.34), the 
transformation yields 


bir Die 
Ce Q11 Gi2 13 
IC] & A222 Ao Dar Oo» 
bai bse 


= ae + Gi12b21 + @13b31  Airbis + Ayob22 + a13b32 461 
Qoibiy + Go2be1 + Ao3b31 Goi bie + Qeebe2 + 23 b30 ( , ) 


Given the same sets partitioned as shown by the dotted lines, 
Clee 11 | Q12 a Thee ee = [Bi] 
EB Ao2 93 = % [4] [42]] [B,] 


=[A, Bi]+[A. B.] (4.62) 
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It should be noted that the last of these is a matrix addition problem. 
The solution in expanded form is then 


b Q1ib Q1ob01 + aizsb31 Ar2b22 + aigbd 
Ce 11011 11 4 12091 13931 12029 13 | 4. 
IC] gue Aoib15 Azebo1 + G23b31 Ge2b22 + aosbse cogs 
bir + Giebo1 + aisb Qiibiz + Ai2b22 + aishs 
Cie 11011 2 31 11912 22 | A. 
we IC] nee + Q2ebo1 + Gosbs1 Gaibie + Aze2b22 + aesbse Ged 


which is the same as Eq. (4.61). 

In general, if a matrix is partitioned into submatrices, the diagonals of 
which are all equal in order (that is, all submatrices are square and of the 
same order), thereby ensuring that all elementary matrix operations can 
be realized, the cells (i.e., the submatrices) can be operated on as if they 
were individual elements of a matrix. For example, consider the follow- 
ing two matrices [A] and [B] in which the individual elements [A;;] and 
[B;;] are assumed to be of equal order: 


[Au] [Aig] [Ain] 
1) Se ane ee 
[Ani] [Ana] [Ann] 
[Bu] [Bir] [Bin] 
and Hind NER ae oon ee (4.65) 
[Bui] [Bro] [Bun] 
The matrix sum is 
Ana Sil [Ae tBu) > [Aw + Bu] 
aera ee 8 ORE (4.66) 
[Ani + Bai] [Anz + Bro] > > > [Ann + Bra] 
Matrix multiplication requires that 
[Cal Cie) ae = [Crs 
LCS ee a ies (4.67) 
[Cai] [Cus] - + > [Can 
where 
[Ci] = [Aa][Bij) + [Ave] [Bo] + > + [Aix] [Bai] 
(jel 2 ered ef el eee ge rT) (468) 


For those cases where [A,;] and [B;;] are not necessarily square or equal in 
order, the required condition for partitioning is that the products and/or 
sums be realized. [Consider, for example, the bordered case of Eq. 
(4.62).] 
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To illustrate the matrix formulation of engineering problems, consider 
the following examples. 


iven the ordinary differential equation 


ce 


= (t) 4.69 
dx dx? ( ) 


subjected to the boundary conditions 


At zx = 0: 
= 1 y’ = (0) 
# (4.70) 
Hip ale 
iS 0) yo == (0) 
in which 
dy d’y 
gees d a 
ody ae dee 
The general solution to Eq. (4.69) is 
y = Asinkx + Beoskx + Cx + D (4.71) 


The problem is to define the constants A, B, C, and D. By substituting 
the boundary conditions into the general solution, the following four equa- 
tions in the unknown constants are obtained: 


Atz =0,y7-= 1: 
A (0) =z B(1) + CO) DO) =a 


ieee = We 
A(k)= -+- — BO) + C1) + D(O) = 0 (4.72) 
Ati l= 0: 
A(sink) + B(cosk) +C(1) + D(1) =0 
Attala: 
A(—k? sink) + B(—k? cos k) + C(O) + D(O) = 0 
In matrix form these are 
0 Le eae eet 1 
oe Ce k ; 7 ee : CEE 
sink cosk O O}| D 0 
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GTI aon 


ZA x 


Figure 4.2 


Example 4.2 


he two-mass spring-connected system shown in Fig. 
4.2 1s SU e 


o a constant force P. The problem is to formulate the 
steady-state expressions relating the displacements wu: and we, the imposed 
force P, the spring constants ki, ke, and k3, and the masses M, and M2. 

By considering the force acting on each of the masses M; and M, as 
represented by individual free-body diagrams and summing forces in the 
vertical direction (see Fig. 4.3), Eqs. (4.74) and (4.75) define the problem. 


koun = k3(ue = U1) + Wi (4.74) 
W, + 7? = 2khiu2 + k3(ue — U1) (4.75) 
Figure 4.3 
ko Us 
W, 
k3(U2— U4) 

kyUp hyp 

We 
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Figure 4.4 


In matrix form, these are 


ke + ks —k3 C3 Wi 
bi Ne oe of se Z| kal . E te tal See 


Example 4.3 To illustrate matrix formulation in transient-type 
problems, assume that the same mass-spring system of Fig. 4.2 is depressed 
a certain amount vertically and then released. Also, assume that a pro- 
portional to velocity damping is present in the system (see Fig. 4.4). 

At the instant the system is released from its disturbed state, the free- 
body diagrams with d’Alembert forces are those shown in Fig. 4.5. 


Figure 4.5 
Cy Uy ko uy 
W, My ui 
k3(U2 U4) 
Coue2 


Ul 
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Figure 4.6 
Again summing vertical forces, 
Kou, — ks(ue — ui) + Mitia + crt, = Wi (4.77) 
2kyu2 + ks(ue — ui) + Motic + cote = We (4.78) 
These can be written as 
U1 U1 U1 ie Wi 

ca] + i [@] + 101[] = [| (4.79) 

_ ket ks —kz or {at,2a0 
where [Ale= | gee, OES: =| [By | 0 WA (4.80a) 

end C1 0 

and (Ch = k | (4.800) 


It should be noted that the coefficient matrix of the w’s, that is, the matrix 
[A], is identical with that of Example 4.2. It is a property of the spring 
system and not of the imposed loading. 


f ‘Example 4.4 A dc network is illustrated in Fig. 4.6. Imposed 
voltages are H; and FE» (directions indicated), and the resistances are Ai, 
R2, ..., Rs. By assuming the unknown loop currents J, 2, Iz, and I4 
and using Kirchhoff’s law—that the net voltage drop in any closed loop 
must equal zero—the following equations can be written: 


Loop I: Hy — Ril, =< R3i =? T3) = Rei a TI») = 0 (4.81) 
Loop 2: —H,. — Rolo — Re(I2 — I1) = 0 (4.82) 
Loop aye —R3(I3 = I) — R,(3 — I4) = (1) (4.83) 


Loop 4: Bere ee va Rl jis 0 (4.84) 
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In matrix form, these equations can be written as 


Rit+k+ Re —Re —Rs; 0 I; Ey 
— te R.+ Re 0 0 I, —E» 
—Rh; 0 R3+ BR, —Ra I; ? 0 

0 0 Slee Ra Rs I EK» 
(4.85) 


Note that the coefficient matrix is symmetrical about the leading diagonal. 
Many engineering problems have equations like (4.85). 


Nis Example 4.5 The three-bar pin-connected space truss shown in Fig. 
4.7 is subjected to an applied force F. The bar lengths are f, f2, and f3. 


Figure 4.7 
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B 
4 
A D G 
ee 
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Q 


oe //- 


Figure 4.8 


Their components in the coordinate directions are listed by the subscripts 
as shown. For example, the component of ¢; in the x direction is f1¢. 
The applied force F has direction cosines a, 8, and y with the coordinates. 
The unknown forces in each of the members are Fi, F2, and F3. (Note: 
Tension has been assumed as positive.) 


2h, = 0 Feosa— Fi(#) + F(@) = 0 


1 £3 
Loy fy 
ZF, =.0 F cos 6 — F.{|—)+ F3(— ] = 0 (4.86) 
Ly £3 
4 Zz £. Zz £ 2 
=F,=0 #£Feosy - r,(#) - F.(@) -#,(%) = 0 


In matrix form, these are 


be _ & |[Fs 
ty Lt; F 
Ont ns ee El | oa. (4.87) 
to 4,1| F 
cos Y 


4, te £3 F 


Note that all quantities have been nondimensionalized. 


‘ Example 4.6 To illustrate matrix formulation in a nonlinear system, 
consider the hydraulic network shown in Fig. 4.8. It is assumed that the 
fluid is incompressible, that the total rate of flow in at A and out at C is 
Q, that the individual rates of flow in B, D, and E are qi, qz, and qs, and 
that the pipe resistances are 71, 72, and rs. It is also presupposed that the 
pressure drop in the direction of flow is equal to the resistance of the 
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pipe times the square of the rate of flow. That is, 
Ap = rqi? (4.88) 
If the rate of flow in pipe ABC is recalled as qi, and that in ADC as qo, the 
rate of flow in AEC will be 
UR = OR 0n2c7 (4.89) 


If the pressure at C is assumed to be zero (i.e., discharge into the air), the 
governing equations will be 


ABC versus ADC: 


Pa eae (4.90) 
ADC versus AEC: 
r2go” = 73(Q — gi — Gz)? (4.91) 
or 
qu*(r1) + q2?(—r2) =/0 
(4.92) 
qi(r3) + qige(27rs) + qe?(rs — T2) + Gi1Q(—2rs) + q2Q(—2rs) = i) 
93 


By nondimensionalizing with respect to Q and defining 


qi 
L=— 4.94 
0 (4.94) 
q2 
and y= — 4.9 
Q (4.95) 
the matrix formulation will be 
x? 
Tae = Fe5 ee 02 ties 0 
Ef 2r3 Uoey 2700} —2rs | ue = head (4.96) 
y 
ni 
J , Example 4.7 Consider the simultaneous differential equations 
| we ar Er es 
dz Fier tates 
a eee (4.97) 
dz ae 
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Figure 4.9 


Writing these in operational form, 


D(x) + 22 — y = 32 


4.98 
Diy) — 3x + y = e* oe 


The matrix formulation is 


D+2 —l #2 32 
Pos” salle] = Le ao 
OV gob 

Example 4.8 To illustrate a case where the resulting matrix is 
readily partitional, consider the transformation of a generalized force- 
couple system in rectangular coordinates x, y, and z to a parallel axis 
system @, g, and Z, which is displaced X, Y, and Z from the original system 
(see Fig. 4.9). For the two to be equivalent, it is necessary that the force 

components and moments about each of the axes be equal. That is 


Vi= Vi Vi = VitZV2— YV3 
Ve = V. Vs = Vs —+- XV3 a ZVi (4.100) 
y= V3 Ve= Vet YVi—-—XV2 


In matrix form 


Vi 1 0 O70 020 11Vi 
V2 0 1 0 10 20404) V2 
Vs aa) Be 0 Peete eet we oe! Vs (4.101) 
Vi 0 +Z —-Y:10 0 Vs 
Vs — J 04. SEXun OU 0] V5 
Ve ee 0" 0. Oost V6 
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Pope 


El Elo Elz 
a b c 
Ea ak ieee 
Figure 4.10 
> [fol 
or vie lenaal [V] (4.102) 
0 Z —Y 
where [A] = 2 0 xX (4.103) 
Yo —X = 0 


Prob. 4.1: Carry out the matrix multiplications indicated in the 


following: 
(a) |}a O Of}ai ade as a (6) UL a 50 | ap ae area: 
O 1 OJ] b: be bs ba O10 by eserpser os 
0 Ot Ci Co C3 C4 OY Oy al Ci Co C3 C4 
. (c) 1 0 0 a1 Ap a3 (d) 0 i 41 Qo a3 ds 
0 1 Qa by be bs ba 0 0 1 by be bs ba 
O° @ Uiee @- @ Ga Om ORR OL RermeCommCcmaC? 
(ec) |}0 1 O|]}a: ae ag as Gf) [O01 auras Sasa. 
It © Win lo y- Wx i) Ol on is, OR dp 
0) @ Wiig @G GG & tO Oe @y @ GG 


Prob. 4.2: A 4 X 4 unit upper triangular matrix is multiplied by a 
4 X 4 lower triangular matrix; that is, [L][7] = [C]. Determine the ele- 
ments of the general matrix [C]. 


Prob. 4.3: A cantilever beam of three segments is subjected to any 
one or any combination of vertical forces P1, Ps, and P3. (See Fig. 4.10.) 
Defining the deflection at the load points in the direction of the loads as 
6;, where 


8: = YP iii (2,7 = 1, 2, 3) 
5 


and 6;; is the deflection of point 7 due to a unit load at point 7, determine 
the matrix equation relating the deflections at points 1, 2, and 3 to the 
imposed loading. 
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Q 
Pig ih er 
=) 
(Alp ye 
le Wa WE 4} 
c 
er 5r or 
L Bip Up 
Q 
Figure 4.11 


Prob. 4.4: For the x, y, and z components of force and moment shown 
in Fig. 4.9, verify that the transformation equations corresponding to a 
rotation of the axis system about the original origin is 


Vi Qzz Azy Az : 0 0 0 Vi 

Vi Age gy Age ! 0 0 0 Vs | 

Ve 24 Qin iy Oz 0 0 0 V3 [R] [0] 

oa Sse SEL i Stee eters =, |b aeene VJ] (4.104 
Va 0) 0 0 Aix Azy Azz V, [0] ak ) ( 
Wee 0° 80 Megs” Op, Og, || Vie 

Ve 0 0.0% a, ay oe LVs 


where a;; is the direction cosine of the angle between the new 7 direction 
and the original 7 axis. 

Prob. 4.5: Using Eqs. (4.103) and (4.104), write the matrix equations 
corresponding to a general rotation and translation of a rectangular- 
coordinate system. 

Prob. 4.6: For the pipe network system shown in Fig. 4.11, assuming 
incompressible flow, discharge into the air, and a pressure drop according 
to the relationship 

Ap = TQ? 


formulate the matrix equation relating flow in each pipe to the indicated 
resistances. (Note symmetry.) 


Figure 4.12 


EA= A= eh= chE oe h= 
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Prob. 4.7: If unknown loop currents J;° in each of the “cells” of 
Fig. 4.12 are assumed is it possible to relate these currents to the voltages 
by the matrix equation 

[A]l7‘] = UI[E] 
where [A] is a function only of the resistances Ri, Ro, ...,Rs? (Note: 
[I] is the identity matrix.) 


Prob. 4.8: Write in matrix form the following equations or expressions: 
(a) Lab = ») Liat eusy (a, b = 1, 7ilp 3) 


(6) Sum of two vectors A and B 

(c) Dot product of two vectors A and B (that is, A - B) 
(d) Cross product of two vectors A and B (that is, A X B) 
(e) Curl of vector A 


4.6 Flow Diagrams for Elementary Matrix Operations 


It is evident from the illustrative examples of this chapter that matrix 
algebra is a powerful tool in the formulation and solution of engineering 


Figure 4.13 
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Figure 4.14 Flow diagram for matrix addition. 


problems. In matrix form, regardless of the area of engineering from 
which the problem arises, the uniformity of the basic mathematical prob- 
lem is most apparent. 

Examples 2.4 and 2.6 of the pivotal-condensation problem of Sec. 3.6 
were concerned with the digital-computer formulation of specific matrix 
manipulations. As in those illustrations, flow diagrams for matrix prob- 
lems invariably require at least one set of nested loops. That is, during 
the course of the computation, each of the subscripts must be incremented 
from its initial to its final value. Because situations of this type are 
observed frequently, either as the total problem or as a segment of the 
problem, it is considered desirable for emphasis to diagram here again a 
general procedure for carrying out such processes (see Fig. 4.13). 


4.6.1 Matrix Addition 
From Eq. (4.23), the defined matrix-addition operation requires that 
Ci = Ay =P bi; (4.105) 
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Figure 4.15 


Both a;; and b;; must be of equal “size.” The new matrix c;; will be the 
same. A possible flow diagram for matrix addition is shown in Fig. 4.14. 
It should be noted that the only difference between Figs. 4.13 and 4.14 is 
the statement contained in the “compute box.” 


4.6.2 Multiplication of a Matrix by a Constant 


The multiplication of a matrix by a constant requires that each and every 
element in the matrix be multiplied by the constant. [See Eq. (4.54).] 
With the exception of the instruction in the compute box, the flow dia- 
gram for this operation is identical with that for matrix addition (Fig. 
4.14). For this second case, the statement shown in Fig. 4.15 should be 
used as the compute box. 


4.6.3 Matrix Multiplication 


From Kq. (4.41), the product of two matrices [A] and [B] is given by the 


expression 
t 


Ci = y inde; (4.106) 


k=1 


It should be noted that three subscripts have been used. This has 
been done to facilitate flow diagramming. It also emphasizes the fact 


Figure 4.16 
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Figure 4.17 Flow diagram for matrix multiplication. 


that the number of columns of matrix [A] must equal the number of rows 
of matrix [B]. 

Replacing the compute box for this matrix problem is the series of 
operations shown in Fig. 4.16, where the index k has been incremented 
from 1 to ¢. The total flow diagram for matrix multiplication is shown 
in Fig. 4.17. 


simultaneous linear 
i algebraic equations 


5.1 Introduction 


An equation of one variable can be represented in functional 
form as 


f(z) =e (5.1) 
The corresponding two-variable problem in x; and zz is 
fi(@i,22) = C1 fo(x1,%2) = Ce (52) 


By extending this system, an n-dimensional set can be written 
as shown by Eas. (5.3). 


fi(@1,22, Shs vers) — 
fo(x1,22, ah eas oy) = C2 

(5.3) 
Tal Gist; ce nee 


In the matrix form of Chap. 4, these are 


[A][X] = [C] (5.4) 


A simultaneous set is considered to be nonlinear if any one 
of the functions f; is nonlinear. Conversely, a linear system 
requires that all functions f; be linear. 
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The solution of systems of simultaneous linear algebraic equations 
requires the determination of the particular values of the variables Bay 
%, ... , & that will satisfy simultaneously the given equations. This, 
however, assumes that such values can be obtained. Consider, for 
example, the following exceptions: 


ue Picts 
( 621 — 32, = 6 
DEPEUDED (5.5) 
= 201 on — 2 
C 
By assuming a value of x; = 1.0, it is observed that the value 
x2 = 0 will satisfy both equations. For x; = 2.0, the value of 22 
is found to be +2.0. An infinite number of other possibilities 
also could be found. 
Hs) v2 
0 | —2.0 
+1.0) 0 
+2.0| +2.0 (5.6) 
+3.0| +4.0 
2. ae 
A (6x1 — 822 = 6 a 
5 Lic HS < 
OW) ari Ti ein head e 


G 


No solution can be found that will simultaneously satisfy these 
(conflicting) equations (5.7). 


It should be noted that in each of the cases demonstrated above, the value 
of the determinant of the coefficients of the z,’s equals zero. 


65k 
oa. 1 


(5.8) 


No unique solutions to the given equations can be found when such is the 


case. 


Certain linear systems behave as if the unknowns (assumed to be inde- 
pendent) are almost dependent. A small variation in the coefficients 
markedly influences the solution to the problem. Roundoff errors also 
have a profound effect. Such systems are frequently referred to as 2l- 
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Figure 5.1 


conditioned systems. Consider the following two simultaneous equations: 


A101 + A12%2 C1 


(5.9) 


Q21%1 + Ao2%e C2 

Assume, now, that the a;;’s and c,’s are not absolutely correct but are 
listed only to a certain accuracy. The true expressions therefore should_ 
be 


(a1 se €11) £1 = (A12 ae €12) Lo —= (i 
(21 a €21) £1 =e (d22 te €29) Xo —= 


where the e’s and y’s represent) error terms. | 

The original equations (5.9) can be represented graphically by the solid 
lines of Fig. 5.1, where the correct solution is given as Vic, Yo. But yi and 
v2 other than zero result in parallel displacements of the two curves, as 
indicated by the dotted lines. For e’s equal to zero and a prescribed accu- 
racy of the c’s defined by the y’s, the solution will lie within the shaded 
region. The influence of the e’s is to change the slopes of the lines and 
thus further alter the region of possible solutions. 

Should the two equations correspond to lines having a small difference 
in slope—that is, should 6 be small—a slight variation in one of the 
unknowns may result in a significant change in the other. Such systems 
are considered to be ill conditioned. It should be noted, however, that 
no fixed angle or function of the angle can be stated that separates an ill- 
conditioned from a well-behaved set. 

The above reasoning can be extended to n-dimensional situations where 
the region of possible solutions is defined by a series of planes. Here, it 
is even more difficult to establish limits of permissible variation. 


For most of the work that follows, it will be assumed that the systems 


ey; 
‘ (5.10) 


ll 


9/2 
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of equations considered are well behaved. The exception to this will be 


the latter part of the discussion of iterative methods of solution. 


-5.2 Cramer’s Rule 


The set of equations under consideration can be represented in summation 
form as follows: 


yy Oyjt; = C; (Gis FO, i) (5.11) 
j=1 


By premultiplying both sides of this equation by a summed set of arbi- 
trary (at this time) cofactors Ai,, Eq. (5.11) becomes 


n 
» An 
t=1 J 


Since the order in which the summation is carried out is not critical, this 
can be written as 


Aizlji = » cA ix (5.12) 
1 


= eal 


n 


y Xj x ajAcnx = » cA x (5.13) 
j=1 oi 


i=1 


But since the product of an element and any cofactor other than its own 
is zero, 
y a;jA ix (5.14) 


t=1 


will have values other than zero only when k = 7. Equation (5.13) can 
therefore be written as 


n 


Lj » aijzAi = >, CAG ai foil, 2% ey 1) (5.15) 


t=1 i=1 


If the process of expansion by cofactors (or minors) is recalled from Chap. 
3, this can be written as 


(2, 0. pan) (5.16) 


PD cae on) (5.17) 


where D is the value of the determinant of the coefficients a;;, and D; is 
the value of the same determinant with the exception that the jth column 
of the a;,’s has been replaced by the column vector ¢1, ¢2, . . . ,¢n. Equa- 
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tion (5.17), known as Cramer’s rule, can be written in expanded form as 


Qi Cis ee C1 Cin 
@o1 Gon °° * Co °° * Gon 
Qni Gna °° * Cn “et Onan 
Lj = (5.18) 
G11 Q12 MU | ae a aR 
Aoi QAo2 Aj Aon 
Ani An2 anj Onn 


To illustrate the solution of a problem by this method, consider the 
following set of simultaneous algebraic equation ; 


33> bicel aeed 
ip 26 |e =| +3 (5.19) 
+2 -1 +3]] 23 \) +1 
For this case Ned 
+3 -—-3 +5 | 
D =) 9437 6 b= 20 (5.20a) 
+2 


Dy= = +20 (5.20b) 
+3 
DS = —40 (5.20¢) 
+2 
+3 
D; =| +1 = —20 (5.20d) 
1-2 
Then 
+20 —40 —20 
=> OC a0) = — = -Z. = — 
(D2 1) 


Before closing this discussion of Cramer’s rule (or, as it is sometimes 
called, solution by determinants), it would be well to consider the “time 
of solution” implications of this method. A 5 X 5 set requires the carry- 
ing out of 6 X 5! = 720 multiplications. Counting just these multipli- 
cations and assuming that the computer to be used can carry out one 
multiplication in 300 X 10-® sec shows that 0.216 sec would be required 
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to solve the problem. While this may not seem to be too large a num- 
ber, consider the rate at which time increases as the number of equations 
becomes larger, as shown in Table 5.1. It should be evident that this 
method is not used to solve larger problems. 


Table 5.1 
Bee Time for multiplications 
equations 
Seconds Hours Days Years 
5X5 0.216 
10 X 10 11.98 x 108 3.32 0.134 
15 X 15 6.277 X 10° 1.745 X 105 (2.7 X& 103 199 
20 X 20 de LLORe1LO** 4.26 X 10” 17.8 X 101° 4.85 X 108 


ae ie 19? 
ow 


5.3 Elimination Methods Ww (97D pT Ck 
eS SS = ---—- r< 
[TAKE 2- Bie 


Methods involving the successive elimination of the unknowns constitute 
the major, direct way of solving simultaneous linear equations. There 


are many variations on this method, and a few of the more important of 


these will be presented. 
STReonG PiAcovat NOT NEELED 
5.3.1 Gauss’ Method Kae a 


To illustrate the basic features of Gauss’ method, the most widely known 
of the elimination methods, consider the general 3 X 3 problem. 


\a 
Ae Co + Gi2%2 + Gists = C1 


~ 


{ we Qo1%1 + Ao2%2 + Ao3t3 = 
18) 


4 
r/ QY Q31%1 + Agee + Ag33%3 = C3 


(5.22) 


| 
s 


By assuming that ai ~ 0, the first of these equations can be rewritten 
with a unit coefficient for x, by dividing the entire equation by au. That 


1S, @ 
£1 + Diete + bists = bis (5.28) 
a a Cc 

where bi = — biz = — bis = — (5.24) 


a 
(A) 


Multiplying Eq. (5.23) by ae: and then subtracting the resulting equation 


from the second of Eqs. (5.22) gives an expression in only x2 and 2s: 


Q22-1%2 + Az3-1%3 = C2-1 (5:25) 


a 
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The subscript notation 22 — 1 is used to indicate the original 2-2 position 
with a new set of coefficients in an n — 1 reduced system. 


Qoi—1) —" oo Aoid12 do3—-1, = Ao3 = Aoibis Cozies Cola Qoibis 


(5.26) 


A similar multiplication of Eq. (5.23) by as: and a subtraction from the 
third of Eqs. (5.22) yields 


Qgo-1%2 + G33-1%3 = C3-1 (Sea) 


- The coefficient expressions are given by 


Q32-1 = Ase — Asibie Q33-1 = G33 — As1bi3 C3-1 = C3 — Asibis 
(5.28) 
Equation (5.25) can now be divided by @z2-1: 
2 + box-1%3 = Des-1 (5.29) 
Ao3— Co= 
where bo3-1 = za and bos_1 = ee (5.30) 
Q22-1 Q22-1 


Multiplying Eq. (5.29) by az2_1 and subtracting the resulting expression 
from Eq. (5.27) yields the following equation in the single variable zs: 


Q33-2%3 = C3_2 (5:38) 
Q33—2 = Ag3—1 — 32-1be3-1 C3-2 = C3-1 — G32-1be4-1 (5.32) 


Dividing Eq. (5.31) by a33_2, 


3 = bg4-2 (5.33) 
' C32 
and : bs4—2 = (5.34) 
Q33-2 


Summarizing, the given three equations (5.22) have been rearranged in 
the triangular form 


41 + dyer, + bist3 = big 
Xe + bes1%3 = Dosa (5.35) 
L3 = des 


It should be noted that, a value for z; having been found by the 
described forward-elimination procedure, values of x, and zx, can be 
obtained by back substitution. Only one unknown will appear in each of 
the calculations. 


An n X n system now will be discussed. It will be assumed that the 


simultaneous linear algebraic equations 73 


equations have the form 


Q@ii1 -- Qiete + Gysl3 + * * * + Aintn = Cy 
Goit1 + Arete + Aost3 + + > > + Gendn = Co 
(5.36) 
Qni®i + Anete + Anstg + **- Ao Oantn = Cy 
For ai; ~ 0, the first of these equations can be written as 
i its Diet 2 Ont, = Din (5.37) 
a1; ‘ 
where bi; = — Fiat (5.38) 
a1 
and m=n+1 
An (n — 1) set of equations in the unknowns 22, 23, . . . , %» can be 


obtained by successive elimination between Eq. (5.37) and the latter 
(n — 1) equations of (5.36). Dividing the coefficients of the first equa- 
tion in this new set by the leading element dz: gives the following 
equation: 

Lo + bes-ivs + °° + + Don—-1tn = Dem-1 (5.39) 


Qoj-1 


where boj_1-= i 2 (5.40) 


Q22-1 


By continuing the process, the leading equation of the (n — 2) system 
can be written as 


23 + bso, + °° + + Dan-2tn = Dam—2 (5.41) 


3j-2 


and b3;-2 = Slee (5.42) 


33-2 


Finally, there will have been carried out (n — 1) eliminations. For this 
case 


Ln = Damtn=t) (5.43) 
Anm—(n— 

where Dam—(n—1) = aaa (5.44) 
Qnn—(n—1) 


Back substitution will yield the other (n — 1) x values. 

Consider the solution by Gauss elimination of the numerical example 
solved in Sec. 5.2 by using Cramer’s rule. The equations are repeated 
as (5.45). 


321 — 3x2. + 5273 = +4 
lz; + 2x. — 6x3 = +3 (5.45) 
221 = 1x2 + 323 = +1 


(onvlue 
am VX 
SYISTE 


(Seu Za 


Pro téEé 


\ 
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The solution is shown in Table 5.2; a parallel part of the table indicates 
the coefficients described above. 


Table 5.2 
ee et 
Line} 21 Xo 23 = const DH X1 Le 23 = const| 2H 
1 ai Q12 13 Ci +3 —3 +5 +4 +9.00 
2 G21 A22 A23 Co +1 +2 —6 +3 0 
3 a31 A32 Q33 C3 +2 —} +3 +1 +5.00 
4 1 bie bis bis 1 —1.00)/+1.67/+1.33 |+3.00 
5 GQo2-1 | G23-1 ¢21(OF G24-1) +3.00|—7.67| +1.67 |—3.00 
6 Q32-1 33-1 | €3=1 (or 34-1) +1. 00|/ —0.34) —1.66 |—1.00 
a 1 bo3s-1 | be4—1 1 —2.56| +0.56 |—1.00 
8 33-2 C3-2(or 34-2) +2 .22 —2.22 0 
9 i b34_2 = 273 1 —1.00 0 


It should be noted that the arrangement of Table 5.2 affords a well- 
ordered system for carrying out the required calculations. To illustrate 
this, consider the first five lines of the left-hand part of Table 5.2, as shown 
in Table 5.3. The element a22_1 in line 5 is obtained by subtracting from 
22 the product of a2: and by2. The elements have been indicated by the 


Table 5.3 
Line x1 Lo X3 = const 
1 ai Qi2 13 Cy 
rrentsS 5 a ee : 
2 i ( Gary | € G22 > Q23 Wie t 
Sa homie ad -- Wee a ey 
3 Q31 a32 A33 C3 
aoe | pated — Got aT 
4 1 Cbis> bis L bis ' 
i eo==== 1 
5 Coo 16 23-1 Come 
pak ec | 
——— ee 
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dashed oval. The element c:_1 would be obtained in a similar manner 
by subtracting from c, the product of ao; and by4. 

The process just described is a continuing one. Line 8, for example, is 
determined from lines 5, 6, and 7, as shown in Table 5.4, which is a seg- 
ment of the original table. The entry in the ‘Constant’ column would be 


(—1.66) — (+1.00)(+0.56) = —2.22 (5.46) 
Table 5.4 
Line 21 2 xs =const | 2H 
5 +3.00 | —7.67 | 41.67 | —3.00 
6 (+1.00!1| —0.34 | f—1.661| —1.00 
7 1 —2.56 | 140.561] —1.00 
8 $2.22 | | —2.221 0 


As is true in all tabular forms of progressive calculations, errors at an 
early stage are not readily apparent. Moreover, they may alter the 
solution materially. To afford checkpoints against such mistakes, it is 
desirable to add another column to the given set. This was done in 
Table 5.2, where the added column was labeled 2H and is the sum of all 
elements to the left in the line in question. By dealing with this added 
column as if it were part of the original system, a check is made when the 
sum of all entries in the line equals that calculated by the above-defined 
process. For example, the 2H entry in line 8 is obtained by subtracting 
from —1.00 (line 6) the product (+1.00)(—1.00). This equals 
(+2.22) + (—2.22) = 0, and a check on the arithmetic is thus carried 
out. Had the two been different because of roundoff errors, it would be 
necessary to make a decision as to which of the two values is to be used. 
The average value is frequently the more desirable one. It should be 
recognized, however, that when any one of the lines does not add hori- 
zontally to that given as 2H, subsequent entry will in all likelihood also 
not check. 

It can be shown that a total of 


5 (n? + 3n — 1) (5.47) 


multiplications and divisions are required to solve a set of n linear simul- 
taneous algebraic equations by the single-division Gauss elimination pro- 
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cedure described above. Since multiplication and division times on a 
computer are approximately equal, these can be lumped together when 
approximate times of solution are being determined. A5 X 5set requires 
65 such operations. If 300 X 10~® sec for one multiplication or division 
is assumed, as was done in the preceding section, the multiplication and 
division time required will be 0.0195 sec. The following table gives values 
for larger-order equations. 


Order of Time for multiplications 
equations and divisions, sec 
5 xX 5 0.020 
10 x 10 0.129 
15 x 15 0.404 
20 X 20 0.918 
40 x 40 6.876 


5.3.2 Cholesky's Method —— ELIM\N ATION 


A variation of the general elimination procedure that can be adapted 
readily to computer usage is that known as Cholesky’s method, or Crout’s 
method. In essence, it consists in the determination of a lower triangular 
matrix that will reduce the original system to the upper unit triangular 
form. 

The original set of equations to be considered is 


[A][X] = [C] (5.48) 
Qi QAi2-°* * * Ain V1 C1 
Qo1 22 Qon Xe Co 

een ag re eee ; " : (549) 
Qni1 Gng2 °° * Ann Xn Cn 


If these can be redefined in the upper unit triangular form of Eqs. (5.50) 
or (5.51), the solution can be obtained by back substitution. 
a ee 


1 tio oF ee Urea U1 ky 
0 1 geerie Xe ke 

ceil emnieeaie (5.50) 
0) 0) ae es 1 Le kn 


[TIX] = [K] (5.51) 
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Since any nonsingular matrix (that is, a matrix having the value of 
the determinant of the coefficients other than zero) can be defined to be 
the product of two other nonsingular matrices, assume that the original 
[A] is the result of a matrix multiplication of a lower triangular and an 
upper unit triangular matrix. That is, 


[L][T] = [A] (5.52) 


This implies, however, that the constants are also related through [L] 
according to Eq. (5.53): 


[L][K] = [C] (5.53) 


This follows from the identity to zero of the two systems, as shown in 
Eq. (5.54): 


[LYTX — K]} =0 =[(AX —C] (5.54) 


Equation (5.54) suggests that for computational work the augmented 
form of the matrices may have advantages. By definition, the augmented 
matrix is obtained by adding to the square coefficient matrix the column 
vector of constants. The augmented matrix of Eq. (5.49) is given as 
Bice (5:55): 


G11 = Q12 Qin : C1 
Qo1 Qoe Gon i Co 
SS okie ee ee ee | : (5.55) 
Ani One Ann : Cn 
ee ae Ce 


By using this notation, Eqs. (5.52) and (5.53) can be written by one 
set of equations: 


G11 Gi2 Q13 | C1 LiOG0 1 tie tis : ky 
Qo1 Qe2 23 Co) = lox Loo 0 0 1 to3 ke (5.56) 
31 G32 33 C3 fz: fs. fs3|{[0 O 1 i: ks 
Augmented matrix Augmented matrix 

[AY Cl = [L] [7 ; K] 


Matrix multiplication can be used to define the entries (,;, t:;, and k; in 
terms of the original a’s and c’s. For example, 


ay = £41(1) e (0) (0) a (0) (0) or t14 = 411 (557) 


c 
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The total group would be 


4 


4. = Qu 


a33 — fsitis — Csotes 
(5.58) 


It should be noted that if the following order of computation is carried out, 
all entries will be defined in terms of previously calculated values. 


1. Calculate first column of f’s; calculate first row of t’s and ky. 
2. Calculate second column of ¢’s; calculate second row of t’s and ko. 
3. Calculate 33; calculate ks. 


The unknown 2z’s are then obtained by back substitution [using Eqs. 
(5.50)]. 


43 = kz 
v2 = ko — tos%3 (5.59) 
ty = ky — ty3x3 — trere 


Since this method assumes an upper unit triangular matrix in all cases, 
space may be saved and a computational advantage realized by elimi- 
nating from the tabulation the 1’s from [7] and writing [7] and [L] in a 
square-set form. However, to emphasize that these are in fact two 
separate matrices, it is suggested that a heavy “‘stair-type” separation be 
shown. Equations (5.56) would be listed as shown in Eas. (5.60). 


G11 Aig A138 C1 


@21 G22 Aas : C2| = (5.60) 


G31 Azo 33 C3 


These can also be shown in the form of Table 5.5. To illustrate the com- 
putational advantage of using Table 5.5, consider the determination of 
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Table 5.5 


(zs) | (Const) 


[A | C] [L\T : K] 


€z2._ Reference to Eqs. (5.58) shows that 
fo2 = G22 — laitie 


That is, f22 is found by subtracting from az. the product of the leading 
row and column elements defining the 2-2 position in the LT set. The 
element 33; equals a3; minus the sum of the paired products (3:13 and 
€zote3. Element t23, on the other hand, is obtained by first subtracting 
from 23 the quantity fsit1; and then dividing the remainder by fx. It 
should be noted that operations of this type can be carried out on most 
modern desk calculators without reentering intermediate values. 

The 3 X 3 systems having been discussed and an ordered calculational 
procedure having been defined, extension to the n X n case can be readily 
made. In general terms 


Bimal: 


lij = Ay — oY lixtes fi = it eS a) (5.61) 
k=1 
1 Gk i 
ae ees 
tii = iz (ai = D, fot) ty = Pai w= (5.62) 


where the order in which calculations are carried out is by rows (for a 
computer) or as shown in the following table (for a desk calculator): 


tin ky first set of calculations 


lon k2 second set of calculations 
ts, k3 third set of calculations 


£n1 
Again consider for illustration the three simultaneous linear algebraic 
equations that were earlier solved by Cramer’s rule and Gauss elimination, 
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that is, 
321 cam 3X2 + 523 = +4 
AIRS) — 1x2 +- 323 = +1] 


A tabular solution is given in Table 5.6. Back substitution yields the 


Table 5.6 


tan 2 3 4 SH 1 2 3 4 DHt 


+3.00/—3.00)+5.00/+4.00} +9.00 ]+3.00 
+1.00)+2.00! —6.00}+3.00 0 EI00 
+2.00]—1.00]+3.00 +5.00 |+2.00 


+1.33) +3.00 
—2.56/+0.56) —1.00 
+2).23|—1.00 0 


+ It should be noted that this horizontal summation is only for the [T:K] augmented 
matrix. Elements of [L] should not be included. Also, the implied 1:00 in the 77 position 
of the [7] matrix must be added when summing horizontally to allow a check. 


following values for the unknowns: 
x3 = —1.00 
Zo = +0.56 — (—2.56)(r3) = —2.00 , (5.64) 
ty = 1.33 = (-F1.67)@3))— (— 1.00) (@) =F 1200 


Again consider the total number of multiplications and divisions 
required to obtain a solution by using Crout’s method. The number of 
these operations required at each stage for a 5 X 5 set is shown in the 
following Table 5.7. 


Table 5.7 


Line it 


ork WN 
SS] oe eS © Se 
ls (SC Wa | on 
Gy TS Co LS aS) 


Back substitution will require an additional 
O-eeteteore's Foto (5.65) 


The total number is then 55 (from Table 5.7) plus 10 from back substitu- 
tion, or a total of 65 multiplications and divisions. This is exactly the 


simultaneous linear algebraic equations 81 


same number as was found for Gauss elimination. A comparison of the 
equations defining the entries in each system indicates that the number 
of multiplications and divisions should be equal, the identical operations 
having been carried out. The advantage of Crout’s method over Gauss’ 
is then the compactness of form and the elimination of entry of intermedi- 
ate steps. This can be seen from Tables 5.2 and 5.6. 

It should be understood that these time calculations are only for 
multiplications and divisions. While, in general, these require much more 
computational time than additions and subtractions, should the number 
of additions and subtractions become large in comparison with the multi- 
plications and divisions, a realistic time evaluation must include both 
items. 


we CAV 


f +0 f 
5.3.3 Square-root Method era OSE CHOLES = Mets 


INSTEAD 
When the coefficient matrix 1s symmetrical, the method due to Banachiewicz, 
known as the square-root method, is one of the most efficient means of 
solving simultaneous linear algebraic equations. Here, as in Crout’s 
method, the augmented matrix [A C] is assumed to be the product of 
two other matrices. These latter calculational matrices, however, are 


presupposed to be an upper triangular matrix [U] and its transpose [U’], 
\ 


[A.C] = [U’[U K] (5.66) 
or, in expanded form, = 

arity, =. ° Ge Os, ti) Ber. 20 He Wis St tel 
Qa Ui2 Ue22 0 0 U22 U2n 
Ani Uin Uo Unn 0 0 Unn 
(5.67) 

(Ail, Ui11 0 Shon 0 ky 

C2 EI, WEY 0 ke 
a | “all ee eee aalaaeiaali 7 AES 

Cn Uin Uon * * * Unn kn 


From matrix multiplication 
Oig = Unilig + Unitas +o Hb With UE) 


Ge = Ai ane ety hha 7 


(5.69) 


Similar equations relate the wu’s, k’s, and c’s. 


= Ce 
WIA 
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By solving Eqs. (5.69) for the values of wu; in terms of the given a’s, 
Eqs. (5.70) to (5.72) are obtained. 


—— a . 
Ui = \/ Gat Uy = a2) (5.70) 
Ui1 
a — a 1 i-1 
ie = J dis — os Tm Tae NO as (ai; = Py Upitty;) et 
(0k) 
and Us = 0 Eo} (or 72) 


The w’s having been defined, the k’s are now readily determined. 
cal 


; 1 . 

et ema 2 (c, = pik) i>] (5.73) 
Ui1 Uti aA 

Back substitution gives for values of the unknown 2’s 


n 


k k; = Dp UipL p 
ln = — and Ly = a Can (5.74) 
Unn Uii 


A tabular arrangement of the type shown in Table 5.8 can be used to 


Table 5.8 
4 1 2 3 4 
El DSH 
(Const) 
Ci U1 U12 U13 ky 
C2 — U3 ke 
C3 U33__ ks 


carry out calculations by the square-root method. Because of assumed 
symmetry, only half the original [A] is given. Similarly, only [U] is 
shown. The u and k values for the illustrated 3 X 3 system are those 
given as Eqs. (5.75a) to (5.75c). 


ae a2 Q13 C1 
U1 = Vai U2 = — hig = ky = — (5.75a) 
U1 U1 U11 
= Q23 — U12U13 Co — Uroky 
Uo, = WV ars — U12? by = ——— ke = —————  (6.75d) 
U22 U22 


jee ye ei rae Cz — (uiski + wosko) (5.75¢) 


U33 
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The unknown 2’s are 


ks ko — u ki — ( x 

a 23% 3 1 (Ui2t2 + Ur13%3) 

oe 7 eee - (5.76) 
U33 U22 Ui 


As a numerical example, consider the following symmetrical, tridiagonal 
set of algebraic equations: 
+4 +42 O}} x21 +2 
+2 +10 +4) ] 272) = 1-46 (Oo. 70) 
O° +4 +5] (23 +9 


The tabular solution is given in Table 5.9. 


Table 5.9 
1 2 3 4 1 2 3 4 ra | 
DHt SH oy ei 
(x1) (x2) (x3) (const) i= 
ie brcka po ea 0 42 +8 14+2.00/+1.00/ 0 |+1.00] +4.00 
2 SiO: eee +6 422 +3, 00|-41.33/-+1.67| +6.00 . 
3 Tos af 414 +1.80/+1.54| +3.34 


t+ Must include in 2H the aj; = a;; terms. | 


L3 = 7 = +0.855 
pe ee aoe = +0.177 (5.78) 
a + 1.00 — ee — (0)(+0.855) = sapere 

The student is urged to verify each of the entries in Table 5.9. 

5.4 Iteration Methods TR Oh © D/A Soy a ae 


The methods for solving simultaneous linear algebraic equations that have 

been discussed thus far require the complete set of calculations to be 

carried out before even an approximate answer to the group can be pe LOUD aaa) ime 

It should also be apparent that these methods are highly sensitive to 

computatio ors. Another feature that may be considered objection- 

able is that little if any computational advantage can be realized for zeros 

that may be contained in the given set when elimination methods are used. 
One means of obtaining an approximate solution to a given system is to 

make an “‘educated guess.” If the first assumed values do not work, 

select others. By carefully noting the influence of each variable, these 
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selections need not be haphazard, and the correct set of values for the 
system of equations can be approached. It should be recognized that 
when this procedure is used, an error in any one guess does not prevent 
the correct solution from being obtained even though a few more trials 
may be required to obtain the necessary accuracy. 

When. guessing follows a prescribed pattern—the assumed values for 
the k + 1 trial being the resulting values of trial k—the method of obtain- 
ing a solution is known as ?zteration. Two iterative methods that will be 
discussed are the Jacobi method and the Gauss-Seidel method. 

For iteration to be successful, the coefficient matrix of the given system 
of equations must be such that successive approximations converge to the 
correct solution. In general, a necessary condition for convergence is that the 
set be diagonal. This requires that one of the coefficients in each of the 
equations be greater than all others and that this “‘strong coefficient” be 
contained in a different position in each equation of the set. (Under these 
conditions, the system of equations could, if desired, be rearranged with 
all strong elements in the 77 position.) A second condition that is suffi- 
cient to ensure convergence exists when the absolute value of the strong 
element in each of the equations is greater than the sum of the absolute 
values of all other coefficient elements in that line. There are other less 
stringent requirements for convergence; however, these are beyond the 
scope of this introductory work. 


5.4.1 Jacobi’s Method —— dace ATION Me THOD 


A strong diagonal set of linear algebraic equations, as represented by 
Kags. (5.79), can be written in the matrix form of Eqs. (5.80), where the 
unknown associated with the strong diagonal element has been isolated 
for solution. 


1101 1+ Gite + Gists °° * = Oink, = C1 
Q2it1 + Agee + Gosts +:°° - > + Gontn = C 
Os. ao O28. 76), AOL 6 we leant Pie (5.79) 
AniX1 ar An2X2 =o An3X3 —- Nee == AnnXn = Cn 
U1 dy 0 bie bis are Din 1 
X2 de bor 0 Dos Don Xe 
x3 ds; bs1 32 0 ae v3 
é = ge | =n n'a Mas mesh lls te ais ee a ame : (5.80) 
Xn dn Dat Das bas coal Soc: 0 Ln 
C; Ai; 
= re and b; = is (5.81) 
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At first glance it would seem that Eqs. (5.80) are inconsistent, 2; appear- 
ing on both sides of the equation. If it is noted that zeros are contained 
in all 27 positions in [B], there is correspondence with the original set. <A 
more meaningful statement of Eqs. (5.80) might be 


a+n[X] = [D] — [Blan [X] (5.82) 


where k represents the number of the iteration and g[X] denotes the zx 
values obtained after k trials. @+»[X] represents the computed (new) 
values of the unknown z’s. 

Consider for illustration the numerical example of Sec. 5.3.3. The 
equations are repeated as (5.83). 


— 441 + 2x2 + Oz: 


STRONG 
D = 2x41 + 10x2 + 423 
[M6 OVAL. aay 


+2 
+6 (5.83) 
Ox: + 4a2 + 5x3 = +5 


ll 


In the iteration normal form of Eqs. (5.80), these would be written as 


X1 +0.50 0 +0.50 0 Li 
x2} = | +0.60 | — | +0.20 0 +0.40 | | x2 (5.84) 


gmltl=> [Dl = [B] @[X] 


It should be noted that the set of equations being considered has a stron 
diagonal. Moreover, the sum of the absolute values of all elements in 

“each row of [B] is less than 1.0. Iteration therefore will converge to the 
correct solution. 


Assume, as a starting point, thatz; = tz. = x3; = 0. (Any other values 
could just as well have been selected.) From Eqs. (5.84) 


%1 = +0.50 x2 = +0.60 x3 = +1.00 (5.85) 


Now, by using these computed values as a guess, a new set of x’s can be 
obtained from Eqs. (5.84). 


x1 = +0.50 — (0.50)(+0.60) = +0.20 
x2 = +0.60 — (0.20)(+0.50) — (0.40)(+1.00) = +0.10 (5.86) 
zs = +1.00 — (0.80)(+0.60) = +0.52 


Table 5.10 is a listing of the values obtained for the three 2’s at different 
stages of iteration. A plot of these values is given in Fig. 5.2. 


N\ETHO D 


5.4.2 Gauss-Seidel Method =, |} TER ATION 


To afford a more rapid convergence of the iterative process, the Gauss- 
Seidel method presupposes the most recent cycle (or part of a cycle) of 
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Table 5.10 , 
eT 
Cycle: First Second Third Fourth Fifth Sixth Seventh Eighth Ninth Tenth 
SS EES 
Zi +0.50 +0.20 +0.450 +0.324 +0.429 +0.376 +0.420 +0.398 +0.417 +0.412 
x2 +0.60 +0.10 +0.352 +0.142 +0.248 +0.160 +0.204 +0.167 +0.186 +0.170 
X3 +1.00 +0.52 +0.920 +0.718 +0.886 +0.802 +0.872 +0.837 +0.866 +0.851 


values at each stage of calculations. For example, assume for a 5 X 5 
set that a value for x3 is being sought for the fourth iteration. Values of 
x4 and 2; to be used in the calculations are those associated with the third 
iteration. Fourth iterative values of x; and x2 would be used. Again for 
illustration, consider the set of equations (5.83) [or (5.84), which is in 
the normal form for iteration]. Starting with assumed values of 
2 = %3 = 0, 


21 = +0.50 (5.87) 
By using this new value for x; and keeping x3 = 0, x2 would be 
te = +0.60 — (0.20)(+0.50) = +0.50 (5.88) 
In turn, 
x3 = +1.00 — (0.80)(+0.50) = +0.60 (5.89) 


21 is now recomputed from the first of Eqs. (5.84), assuming x2 = +0.50 
and x; = +0.60. The solution is summarized in Table 5.11, and the 
values are plotted in Fig. 5.3. 


Table 5.11 


Cycle: First Second Third Fourth Fifth Sixth Seventh Eighth Ninth 


21 +0.50 +0.25 +0.345 +0.384 +0.401 +0.408 +0.411 40.412 40.413 
x2 +0.50 +0.31 +0.231 40.197 +0.183 +0.177 40.175 +0.174 40.173 
x3 +0.60 +0.75 +0.815 +0.842 +0.854 +0.858 +0.858 40.861 +0.862 


Function value 


Cycles of iteration 
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Function value 


Cycles of iteration 
Figure 5.3 


5.4.3. Iteration of Nondiagonal Sets 


Before leaving the discussion of iteration methods it would be desirable 
to observe the behavior of a set that is not diagonal. Consider the equa- 
tions that were solved earlier in this chapter by Cramer’s rule and by 
Gauss elimination. 


821 — 3x2 +523 +4 


la, +202 —6x3 = +3 (5.90) 


2%, —I12z. smd al 


In the normal form for iteration, these are written as 


L1 +1.33 0 = 1.00) 1567 (iar 
x2| = | +1.50] — | +0.50 0 —3.00 | | v2 (5.91) 
L3 +0.33 Oe 0.05 0 XB 


Using the Jacobi method, the first nine iterations are summarized in Table 
5.12 and plotted in Fig. 5.4. The correct solutions are 7; = +1.00, 
2. = —2.00, and x3 = —1.00. It should be recalled that iteration is not 

— 


recommended for these types of problems. 


Table 5.12 


_ Cycle: First Second: Third Fourth Fifth Sixth Seventh  Highth Ninth 


a. +1.33 +2.283 +3.425 42.057 42.765 +1.474 +2.023 -—6.025 +6.909 
ze +1.50 +1.825 —0.243 -—1.953 -—5.622 —4.950 —7.915 -—0.514 —6.450 
zs +0.33 —0.160 —0.580 -—2.031 —1.689 -—3.384 —0.334 —3.654 +4.179 


5.5 Relaxation Methods 


The method of relaxation (like iteration) differs from elimination in that 


solutions for all unknowns are obtained simultaneously. However, unlike 
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+10 eres 
[ fT Jacobi method —+—} + 


Nondiagonal set of equatio 


+ 
oO 


Function value 
eo) 


| 
(6) 


10 
Figure 5.4 


iteration, it is impossible in relaxation to prescribe at the outset a position- 
fixed order by which the various terms in the solution are to be “‘adjusted.”’ 
Another variation from the iterative procedure is that relaxation usually 
is concerned with manipulation of the amount of “‘out-of-balance,”’ rather 
than with the entire magnitude of the quantities in question at each stage. 

The set of linear simultaneous algebraic equations to be considered is 
defined by Eq. (5.92). 


[A][X] = [C] (5.92) 

These equations could have been written equally well as 
[A][X] — [C] = 0 (5.93) 

or in the summation form ; 
», Ayjli — Cy = 0 (2 = 12 2, 5) 5 Sy n) (5.94) 


j=l 

For the correct solution of the 2’s, the equations will be satisfied iden- 

tically. Deviations from these correct values, however, result in other 

than a “zero” correspondence. To facilitate a solution by the method of 
relaxation, Eqs. (5.94) will be written as 


[A][X] — [C] = [R] (5.95) 
or >») Ait; -— = R; (a = 1, 2, a 85 F n) (5.96) 
j=1 
or Q11%1 + Ayo. + Arst3 + °° + + Aintn — ¢, = Ry 
Qoi%1 + Aoo%2 + Aostz + °° > + Conta — Co = Re 
(5.97) 


eer ee Ot Th ety ety MRM MT TOT. O (heach &% ot OG GO who 


simultaneous linear algebraic equations 89 


The object is to alter the assumed 2’s so that the R’s, the residuals, approach 
zero. 

The residuals, as shown by Eqs. (5.95) to (5.97), are functions of the 
variable x’s. That is, 


R; = R;(21,X2, A evo ons) (5.98) 


A change in the value of any one or all of the z’s will alter the R’s. By 
considering only the first term of the series describing the change in R 
with x, the total change in the residuals (that is, the differentials of the 
k’s) will be those given by Eqs. (5.99). 


oR oR 
dk, = ed dx + ial dx. + ae oe + ans dz, 
021 OX On, 
oR OR fe) 
dpa — dee — dry + 2 oe dey ee 
O21 0x2 OXn (5.99) 
OR, OR, 0 n 
ON dz; + dzz +--+: + + Ht az, 
021 0X2 OXn 


=a; (@j=1,2,...,n) (5.100) 


Equations (5.99) can therefore be written as 


dR, = G11 dx, + Gig dao + - - > + Ain dtp 
AR. = G21 dx; + Qe2 dt2 + > - + + Gon din 

(5.101) 
dR, = An1 dx + An2 dx + rive *y sie Ann din 


The relative influence of each of the dz;’s on the residuals R; now can be 
obtained by selecting, in turn, a unit change in the value of one x while 
all the others are kept unchanged. The corresponding unit-relaxation 
table can be set up as shown in Table 5.13, where in order to represent 
the finite increments involved, deltas replace corresponding total differ- 
entials in the R’s. It should be noted that the entries in the AR portion 
of Table 5.13 are the transpose of the original a;; coefficient matrix. 


5.5.1 Relaxation of 3 x 3 Set of Equations 


To illustrate how relaxation is carried out by using Table 5.13, consider 
the following set of equations. These are the same equations that were 
~ considered in Sec. 5.4, Iteration Methods. 


90 modern methods of engineering computation 


Table 6.18 Unit-relaxation Table 


1 0 0 aun a21 Q31 Ani 
0 1 0 Qi2 22 32 Anz 
0 0 1 Gin te Aan Gnn 
Avie eas = +2 
2%. +102. +423 = +6 (5.102) 
4%. +623 = +9 


The corresponding unit-relaxation table is shown as Table 5.14. Since 
the given set is symmetric, the AR values in this table are also symmetric. 


Table 6.14. Unit-relaxation Table 


The standard form of the equations for relaxation is given by Eqs. 
(5.97). For the set in question, this is 


Aq, +222 —2= R, 
Ar +523 —5 => Rz 


As a starting point, select 71 = x. = x3 = 0. (Any other values could 
have been presumed just as well.) The residuals areR; = —2,R. = —6, 
and R; = —65. Since the object of the relaxation procedure is to adjust 
the x’s successively so that the R’s approach zero, and since Re is the _ 


residual of greatest absolute magnitude, it is necessary first to determine 
which variable has the greatest influence on R, and then to _to determine 
how much that variable should be : be adjusted.) From Table 5.14, it is 
observed that a unit change in 22 will have the greatest influence on Ro, 
altering the value of R. by a factor of 10. Therefore a change of 10.6] 
from the assumed value of x2. = 0 will bring the residual R, to zero. At 


the same time, it will change the value of the residuals R; and R3. 


iS oe Soe Vi eae os 
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Rea 2-4 C\0.6) "208 
Rz = 8°G+\ 9 (06) = (5.104) 


Rea +\(0.6) ay 6 


A second correction is now in order. Since Rs is the greatest residual and 
since a variation in 2; will have the greatest influence on Rs, a change in 
x3 from zero to a value of +0.52 will reduce R; to zero. The residuals 
are now 


Ri = -08 +] 0/652) = ~98 
R, 0 +|(4)(0.52) = +2.08 (5.105) 
Rs = —2.6 +|(5)(0.52) = 0 


At this stage, then, the approximate solutions with corresponding residuals 
are 


ri2= 0 = 0 Ri = —0.8 
t2=0 +06 = +0.6 R. = +2.08 (5.106) 


Table 5.15 summarizes the solution of the set of equations using this 
method. In all cases, the residual being reduced is underscored. 


Table 5.15 


1 

2 

3 

4 

5 

6 

“ 

8 

4) 
10 0 0 +0.04 0 +0.30 +0.03 
11 0 —0.03 0 —0.06 0 —0.09 
12 0 0 +0.02 —0.06 +0.08 +0.01 
13 0 O01 0 —0.08 —Onoz —0.03 
14 +0.02 0 0 0 +0.02 —0.03 
15 0 0 +0.01 0 +0.06 +0.02 
16 0 —0.01 0 —0.02 —0.04 —0.02 


Solution 
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To check that no arithmetic mistakes have been made, substitution of 
these values in Eqs. (5.103) yields 


(4)(+0.41) + (2)(+0.17) —2 = R, = —0.02 
(2)(+0.41) + (10)(+0.17) + (4)(+0.86) —6 = R, = —0.04 = (5.107) 
(4)(+0.17) + (5)(+0.86) —5 = R; = —0.02 


A consideration of Table 5.14, the unit-relaxation table for the problem 
in question, indicates that a unit variation in 2; has its greatest influence 
on R,, only half that value on Re, and no influence whatsoever on R3. 
The variable x. manifests itself most forcibly on Re, Ri and R; being much 
less affected. However, a unit change in x; has about the same influence 
on R. as it does on R3. This latter situation is not too desirable for 
efficient relaxation. By adding, subtracting, and/or multiplying the 
individual rows of that unit-relaxation table, this situation can be 
improved and a more efficient operation table developed. A possible 
resulting group-relaxation operation table for the problem just considered 
is shown in Table 5.16. The entries in this table are obtained by multi- 


Table 5.16 
Az, Ax. Azz AR, AR, AR3 


ap ll 0 0 +4 +2 0 


Oiuecitls me ty geo ae 10RN ees 
(ee a Pe So =2iee=b6 


plying row 3 of Table 5.14 by 2 and then subtracting row 2 of the same 
table from it. By using this group table, the solution is that shown in 
Table 5.17. Three relaxations using the group table 5.16 result in the 


same order of accuracy as did 16 relaxations using the unit-relaxation 
Table 5.14. 


Table 5.17 


1 0 Sa OmG) 0 SU 0 —2.6 

2 0 —0.43 +0.86 ROG —0.86 —0.02 

3 +0.42 0 0 +0.02 —0.02 —0.02 
Solution +0.42 =). te +0.86 +0.02 —0.02 —0.02 
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9.5.2 Relaxation of Larger Sets of Equations 


A more realistic type of engineering problem might require the solution 
of the following set of equations: 


1.621 +2.0%2 +0.8%, —1.22, = +37.2 
1.87, +0.7%. —1.6%3 —O0.82, = +42.0 
2.0%, —1.6x%2. +0.423 —3.0x, = +13.8 
1.2%, —14z. +1427; +2.02, = —30.4 


(5.108) 


Since any or all of these equations could be premultiplied throughout 
without changing their values (that is, they could be scaled), the equations 
will be written in the relaxation form: 


16z; +202. +84; —12%, —372= Ri 
18%1 +72. —1623; —82, —420 = R, 
20z1 —l6z, +4273; -—302, —138 = R; 
127, —14%. +147; 4202, +304 = KR, 


(5.109) 


The unit-relaxation table is given as Table 5.18. 


Table 5.18 Unit-relaxation Table 


Az; Axe Ax3 A%4 AR, AR» AR3 AR, 
1 0 0 0 +16 +18 +20 +12 
0 1 0 0 +20 +7 —16 —14 
0 0 1 0 +8 —16 +4 +14 
0 0 0 1 —12 —8 —30 +20 


It should be noted that none of the elements are particularly strong in 
this relaxation table. All values are approximately equal and, should 
relaxation be undertaken using them, convergence would be relatively 
slow. The development of a group-relaxation table is, therefore, highly 
desirable. If only unit combinations of the above are considered, Table 


Table 5.19 Group-relaxation Table (1) 
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5.19 isa possible group table. A “stronger” group table, however, is given 
in Table 5.20. It will be used in the solution that follows. 


Table 5.20 Group-relaxation Table (2) 


It should be recognized at the outset that when relaxation techniques 
are used to solve linear simultaneous equations, accuracy can be improved 
at any stage by simply carrying arithmetic operations to more digits. 
Therefore, at an early stage, little is to be gained by finding “exact” 
multiples of the x’s to counterbalance given or resulting residuals. As 
will be demonstrated in the solution that follows, greater accuracy and 
ease of solution can be achieved by operating on multiples (by factors of 
10”) of the unknowns and corresponding residuals. 

The solution is summarized in Table 5.21. The solution, to the 
accuracy thus far obtained, is 


%i = +84,734 X 10-* = +4+8.4734 
%. = +115,090 * 10-* = +11.5090 
(5.110) 
x3 = —85,665 X 10-4 = —8.5665 
t, = —62,313 X 10-*= —6.2313 
with residuals as follows: 
R, = —20 X 10-4 = —0.0020 
R, = —14 X 10-4 = —0.0014 
(5.111) 
R; = +10 X 10-4 = +0.0010 
= —0.0022 


R, = —22 X 10-4 


Prob. 5.1: Obtain the solution to the following set of equations by 
the Gauss, Cholesky, square-root, iteration, and relaxation methods. 
(Carry the solution to three decimal places.) 
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Table 6.21 Solution of Eqs. (6.109) Using Relaxation Table 5.20 


—————— 


Linet Ly, Lo v3 4 R; Re R; R, Note 
i oe —l00 anlar —a OMe — Oe |l Sei —420) 138i) 304 
2 +12 —1§ +6} —396|} +86] —150) +316 
3 +6 12 +12 +36 | +86 | —174| +388 
oa —9 +3 —9 —9} —12/ +111] —168| —68 
5 =3 +1 —3} —28]} +98} —26| —72 
6 —2 +3 —l) —24]} +22) -—24| —74 
6’ +7 +12 —1ll1 —7| —24) +22) -—24|) —74 
Tt +70 +120 —110 —70] —240 | +220} —240| —740} «107 
8 +15 —5 +15 +15] —160} +95) —250|) +20 
9 —6 +2 —6} —192) +69} +384] +12 
10 +3 +6 +6 +24] +69} +22) +48 
il —2 +3 —lj +28 —7|} +24/] +46 
Ta +86 +115 — 84 —62] +28 —7| +24) +46} x10"! 
12 +860) +1,150 — 840 —620}] +280} —70} +240) +460] «10-2 
13 —9 +3 —9 —9} +-232 +5 | +246 +4 
14 —3 —6 =—6 +16 +5] +258) —32 
15 +6 —2 +6] +48) +381] —26| —24 
iy +848) +1,153 — 857 —623]} +48] +31] -—26] —24] x10? 
16 +8,480) +11,530) —8,570| —6,230] +480] +310] —260| —240| x1073 
ily/ —7 —14 —14 —24| +310] —232 | —324 
18 +6 —2 +6 +6] +8] +260] —236| —20 
19 —8 12 —4) +24} —44}] —224|} —28 
20 —6 +2 —6 —8| —70} +60| —36 
74 +2 —3 +1} —12 +6} +58} —34 
il +8,473) +11,508) —8,567}| —6,233} —12 +6) +58] —34| 107% 
22 |+84,730/ +115, 080] —85,670| —62,330] —120} +60) +580] —340| 10-4 
23 +12 —4 +12] —56/+112| +12) —324 
24 +6 —2 +6 +6) —24] +62 +8] —20 
25 —2 +3 —1} —20} —14] +10] —22 
25’ |+84,734|+115,090)—85,665|—62,313} —20| —14/} +10] —22 10m 


ee | ee ee ee Se SS 
+ Lines designated by primed numbers summarize solutions obtained to that point. ; 
t Since the values of the residuals are small in comparison with the operators listed in 
Table 5.20, it is desirable to multiply all quantities by a factor of 10 before relaxing further. 
That is, f;(1021, 1022, 1023, 1024) = gi(10Ri, 10R2, 10R3, 10R4). The corresponding set 1s 
given by line 7. ; 
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+141 —I1x, = +1 
—la1 +22, —Ia; = +2 
—lx. +3273 —1%, = +3 

—lxz3; +22, —l1lxv5 = +2 

—la, +1xz; = +1 


Prob. 5.2: Solve the problem in Example 4.4 for 1, I2, I3, and 4 in 


terms of LH, where 
Ry = R Ey, = fis} 


R,= Rk E, = 2E 
R3.= 
Ry, = 2k 
R= RK 
Re = 3h 


Prob. 5.3: Using Cholesky’s method, solve the following set of 
equations: 


621 + 322 = +36 
341 +1622. + 5zs = +120 
ote +1473 +22, = —124 

| 2r3 +42, = +2 


Prob. 5.4: By solving the following 3 X 3 set of equations by the 
square-root method, verify that no special difficulties are encountered 


when uj is the square root of a negative number. (Hint: V —a* = ai.) 
4e1 + 2x = +2 
2%, —107%. +42; = +6 
4t2 +523 = +5 


Prob. 5.5: If a digital computer does not recognize the difference 


between V —a? and V +a’, will a solution by the square-root method be 
usable? Why? 


Prob. 5.6: Solve, by relaxation methods, the following set of equa- 
tions [Note: Eq. (5.21) and Table 5.2]: 


321 — 32 +523 = +4 
la; +2%5 —623 = +3 
27, —lx, +3823 = +1 
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Prob. 5.7: Solve, by any of the presented methods, the following sets 


of equations: ae, 2 Meee aes py eeeel ZS 
OU s sale +47¢ ; poe =2| Se? | A 
—2 0 -2 Le) =} +2 eee nw = mat 
+1 42 O]la, Se 


Ly +8 

C2 — ape 

X3 +4 
100 24-8) 9} | aa +35 
eee Pee ON ere | =| — 25 
—13 —14 +12]|2; 
(dq) |} —3.2 +46 443][a +2.0 
4.2, -+-3:2 2.8) | 22.) =) 4+44 
+0.6 —0.8 +0.4]] 23 


2 fhe De 45) 1 x; 0 
+3 +20 +6 —4]/I 2. +2 


(i) ale 10 0 42 —-1|/Fa 0 
+2 +12 +43 —4|/a2.|_ | O 
+1 -4 +18 +6]]23/ | +1 
0 -4+3— —3 —12 | le, =| 

(gy [+1.8 +1.2 0 0 r1 9:8 
Pigeeeos io 70 v,| | +1.0 
Ome ieee A is | tT 
0 Oo F128) =-3:0:} an abi 


Prob. 5.8: Solve Example 4.1 for values of k = 0, 7/2, and zx. 


Prob. 5.9: Show that the product of the elements ¢;; is equivalent 
to |A|. 


5.6 Computer Solutions 


In selecting a method of solution for digital-computer programming, two 
factors are normally considered. ‘The first of these is concerned with the 
ease of programming; the more repetitive the calculational process, the 
easier is the programming. Secondly, the running time of solution is a 
major factor. For smaller computers, a third consideration, and the one 
that may be the controlling factor, is storage capacity. 

Flow diagrams for two different types of solutions will be illustrated in 
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Enter all required constants and coefficients, ¢; 
and a;;. Set all xj's to zero, and initialize any 
other variables. 


With both / and / ranging from 1 to a, calculate 
the 7 values of x; from the following equation: 


If the set of x's just obtained is the first set 
calculated, save them, go back to Step II and 
compute a second set based on the first. 
Compare each of the x;'s. lf the absolute value 
of the difference in any one of the comparisons 
is greater than a prescribed amount, save the 
last computed set. Return to Set II, compute a 
new set of x's and again compare the results 
with the assumed values. Continue this process 
until the maximum deviation between two con - 
secutive sets of values is within the error allowed. 
Put out the current values of the x's. 


Step Ill 


Gauss-Seidel method 


Figure 5.5 


this section. The ones that have been chosen for discussion are the 
Gauss-Seidel method (iteration) and the Cholesky method (elimination). 


Good Fasl PROGRAM 


As was demonstrated in Sec. 5.4.2, the solution to sets of linear equations 
by iteration requires the repeated execution of a given elementary com- 
putational process. Moreover, the results of any one set of calculations 
depend on the last preceding set of calculated solutions. Such systems 
are ideally suited to flow diagramming, as will be apparent from the fol- 
lowing discussion. 

In broad outline, Fig. 5.5 is a flow diagram that could be used to obtain 
the solution to sets of simultaneous linear algebraic equations by using 
the Gauss-Seidel (iteration) method. A more detailed consideration of 
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the specific steps necessary to carry out these instructions is given in 
Figs. 5.6 to 5.8. The complete flow diagram is shown in Fig. 5.9. 


Strep J 


The flow diagram in Fig. 5.6 will enter the data and initialize the variables. 
With regard to the notes indicated in Fig. 5.6: 


1. The coefficients of x, that is, a;,, the values of c;, and the number n 
are read into the computer. ‘Error’ is a term which represents the 
maximum allowable difference between corresponding values in any 
two consecutive sets of computed z;’s. ‘Error,’ then, represents 
the precision to which the values of x are to be computed. For 
example, if the values of x are desired to five decimal places, the 
prescribed error is 0.000005. 

2. kis a constant used to identify the first calculated values of the z’s. 
This is needed since the first set of values is to be saved without 
making comparisons, whereas each subsequent set must first be 
compared with the set immediately preceding before being used. 
k will therefore need to be changed after the first set of values has 
been obtained. 


Strep IT 
With both 7 and j ranging from 1 to n, calculate the n values of x; from 
the following equation: 
1 n 
Gy ES = (c, — > aist;) (5.112) 
Ay ey 
ji 
Figure 5.6 


Notes 
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Figure 5.7 


This part of the flow diagram is shown in Fig. 5.7, with notes as follows: 


1. All values of a;; and 2; are to be used in the calculation of any 
particular x; except 2; itself. The computational block noted as @ 
is used to identify that particular entry to be excluded. If7is equal 
to 7, that value of 7 is skipped and the next value used. However, 
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before this is done, a check is required to determine whether 7 has 
become as large asn. If it has, no additional sums are accumulated. 
If not, j is incremented by 1. 


2. A particular value of 2; is calculated at this point. Sum contains 
the products of all current values of a,; and 2; (with the exception 
of the 7th terms). 


3. At this stage, it is necessary to determine whether all n of the values 
of x have been calculated. If they have not, 7 is incremented by 1, 
j is reset to 1, and the process is repeated for a new 2;. If 7 does 
equal n, all the x,’s have been determined and computation proceeds 
to the next step. 


4. If the rate of convergence to a final set of x’s is slow, it may be 
desirable to have the computer put out at any time the most recent 
set of values. Manual control of most computers, through various 
sensing mechanisms, can provide this feature. 


Srrep III 

If the set of z’s Just obtained is the first set calculated, save them, go back 
to Step II, compute a new set of x’s, and again compare the results with 
the assumed values. Continue this process until the maximum deviation 
between two consecutive sets of values is within the error allowed. Put. 
out the current values of the z’s. 


This part of the flow diagram is shown in Fig. 5.8, with notes as follows: 


1. If at any time in the computation the rate of convergence is found 
to be too slow, the process can be accelerated by jumping ahead to 
new values of x; This block allows for the introduction of such 
values. The mechanics of such introduction is much the same as 
for an intermediate output. 


2. At this stage in the calculations, it is necessary to determine whether 
the computed set of values of z; is the first (that is, k = 1) or some 
later set. (It will be recalled that in Step I the variable k was intro- 
duced to facilitate comparison between the first and second sets of 
computed values.) Immediately after identifying the first set, the 
value of k is changed to zero, and subsequent exits are through the 
path marked “No.” Each of the values of z; is placed in a tempo- 
rary location called sz; When a complete set has been stored, the 
entire process is repeated (@®) and a new set is computed. 

3. Each set of values of x is compared with its corresponding values in 
the previous set, that is, sz;, to determine whether any of the indi- 
vidual differences exceed in magnitude the initially prescribed error. 
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Figure 5.8 


If for any of the z;,’s the difference is found to be greater than that 
allowed, the last computed set is saved in the sz; location and a new 
set of x;’s is computed. When all the values of x; satisfy the error 
test, these values are put out as the final answer. 


Figure 5.9 is a complete flow diagram of the Gauss-Seidel method for 
solving simultaneous linear algebraic equations. 


5.6.2 Cholesky’s Method 


As is true of all the elimination procedures, Cholesky’s method reduces 
the given set of equations [A][X] = [C] to the upper unit triangular form 
[T ][X] = [K]. Back substitution then can be used to produce the required 
values of x. | 

Reduction to the upper unit triangular matrix is accomplished by use 
of a lower triangular matrix [L]. Since one of the computations that 
must be carried out is a division by ¢;; [see Eq. (5.62)], the major diagonal 
element of [Z], it is necessary that this element be other than zero. If a 
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Figure 5.9 Flow diagram for Gauss-Seidel method—solution of 
simultaneous linear algebraic equations. 
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Figure 5.9 (Continued) 


zero is found to exist, rows must be rearranged in such a way as to elimi- 
nate the zero before solution proceeds. 

A very general flow diagram for the solution to this problem is shown 
in Fig. 5.10. Detailed flow diagrams of the various segments of the 
general program are shown in Figs. 5.11 through 5.17, and a complete 
diagram is shown in Fig. 5.18. 


Srp I 
Read in the values of n, the coefficients a,;, and c;, Form a new matrix, 
which has the dimensions n + 1 by n, by adding the column vector c; as 
the (n + 1)st column of the existing a;; matrix. Check a; for zero. If 
it is not, proceed, If ai: is zero, check values of a; until an element in 
the first column is found which does not equal zero. Exchange that row 
with row 1. 

A detailed flow diagram is shown in Fig. 5.11 (see p. 107). The notes 
indicated in the figure have the following significance: 


1. The value of n read in at this point is the size of the set of equations 
to be solved. a,; and c; are the coefficients of the unknown 2’s and 
the constants, respectively. 

2. The coefficient matrix is augmented by adding to it the column 
vector of constants c;, These are placed in the (n + 1)st column 
position. 

3. In calculating the elements of the first row of [T], it is necessary 
to divide by the value of a1. Thus, ai cannot equal zero. If it 
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Read in the values of 7, the coefficients ajj and cj. 
Form a new matrix, which has the dimension 7 +1 

by ”,by adding the column vector cj as the (7 +14)st 
column of the existing a7; matrix. Check a,, for zero. 

If it is not zero, proceed. If 4, is zero, check values 

of a;4 until an element in the first column is found 

which does not equal zero. Exchange that row with row 14. 


Compute the first column of lj; the first row of rij, and 
the diagonal 7/7. Note that A; is the last column of f;;. 


im 


Using the equation 


= 
Lip = O77 -) bin ‘nj 
=4 
compute the elements of the /th row of[ZL]up to Z;;. 


ee 


Check Z;; for zero.\f Z;; equals zero, go to Step VII. 
lf Z7; is not equal to zero, calculate the elements of 
tj from the equation 


j= 


ae ey at 7 : ; 
tj =F ee ~) tt tj 
k=1 


Values of / are to range from 4 to 7. The index / 
varies from / +1 to a +14 (thereby including £; ) 


Ze 


Increment / by one and compute the elements of [4] 
and [7] for next row. Continue this process until all 
values have been calculated. 


ee 


Using the computed values of 7// and noting that : 


L 


ti “7 +4 8 actually *;, back substitute and compute all 


values of x;. These values of x are the final answers. 


has a value other than zero, the program proceeds. 


Figure 5.10 


If, however, 


ai1 is found to be equal to zero, a1 is checked to see whether it 
equals zero. Then dz: is checked, and so on until a value of aj is 
found which is suitable. If a nonzero a;1 cannot be found, the prob- 


lem stops. 
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lf in Step IV Z;; = O, go back to a;4 and save 
all of the /th row.Move the (/+1)st row and 
all succeeding rows up one position. Place the 
row previously saved into the th position. Go 
back to Step III and recompute a set of qij's 
for the /th row. 


==Aifya = 


Figure 5.10 (Continued) 


4. This check is to determine the current value of z and thereby to 
identify the row for which a;, was not zero. If the current value is 
1, the problem proceeds to Step II. If the current value of 7 is 
not 1, the 7th row is interchanged with the first row. 


Srrp II 
Compute the first column of ¢;;, the first row of t,;, and the diagonal t,. 
Note that k; is the last column of ¢;;. 

The following are noted in Fig. 5.12: 


1. In Cholesky’s method, [Z] is a lower triangular matrix, the first 
column of which is equal to the first column of the matrix [A]. 

2. [T]is an upper unit triangular matrix. This step in the flow diagram 
sets the diagonal elements of t,; (that is, ti;) equal to unity. 

3. The first row of t;; equals the first row of a,; divided by ay;. How- 
ever, this step is begun at 7 = 2, since ty: has already been set to 
unity. When all values of t:; have been calculated, including t1,n41, 
computation proceeds to Step III. 


Strep III 


Using the equation — 
j=l 


Li; SU Linbats 
2 : 


compute the elements of the 7th row of [L] up to ¢,. 
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Notes 


Figure 5.11 
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Step II 


Figure 5.12 


That part of the total flow diagram which causes this step to be accom- 
plished is shown in Fig. 5.13. Each of the steps is self-evident, with the 
possible exception of the ones noted. 


1. Index 7k is a variable used to keep account of the current value of 7. 


2. Since 7 is initially set at 2, £:2 is the first element of ¢;; to be deter- 
mined. (Note that f;; has already been defined.) When 7 has been 
increased to 2, the problem passes to Step IV. 


In each row, values of f;; beyond ¢;; are zero. 


Strep IV 
Check ¢;; for zero. If ¢;; equals zero, go to Step VII. If ¢;; is not equal 
to zero, calculate the elements of ¢,; from the equation 


Cal 


n iy = = (a, — » Cixtns) 


k=1 


Values of 7 are to rangefrom1lton. jvariesfromi + 1ton + 1 (thereby 
including k;). 
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Notes 


Step III 


Figure 5.13 


With regard to the notes indicated in Fig. 5.14: 


1. Since at this stage division by 4;; is required, it is necessary to check 
whether ¢;; equals zero. If not, the computation proceeds. If, 
however, it is equal to zero, control passes through exit @) to a special 
routine, the purpose of which is to eliminate that zero (Step VII). 

2. When entering this part of the flow diagram, the value of 7 was the 
same as7z. Since ¢;; has already been defined, 7 is set at 7 + 1, and 
tii11 is the first element computed. In succession, each element 
for that row is found, up to ¢:,n41 (which is actually k;,). 


Strep V 
Increment 7 by 1 and compute the elements of [L] and [7] for the next row. 
Continue this process until all values have been calculated. 

Regarding the notes in Fig. 5.15: 


1. At this stage, only one row of ¢;; and t,; have been found. This part 
of the flow diagram causes each of the remaining rows to [L] and 
[7] to be calculated. 
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Notes 


J=zl+ 


wise 


Sum = Sum + ty; x; 


Step VI 


Figure 5.16 


2. ik is a variable used to keep account of the current value of 7. It is 
not used unless f;; is found to equal zero, in which case the rows must 
be interchanged. 

3. @ is the point in Step III (see Fig. 5.13) to which the problem 
returns to compute a new row of [L] and [7]. 
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Notes 
\ 
2 
3 
4 
TA= L444 
Zn4 = Savel 
/k =k +4 5 
/ = temp 


w Step VIL 


Figure 5.17 
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Strep VI 
Using the computed values of ¢;; and noting that t,n41 is actually k,;, back 
substitute and compute all values of x;. These values of x are the final 
answers. 

The notes in Fig. 5.16 correspond to the following: 


1. At this stage, each element in the last column of the augmented 
matrix ¢,; is placed into a special location called k;. These values 
are used in the back-substitution routine with the equations 
[TI[X] = [K]. 

2. Since ¢,; is an upper unit triangular matrix, the nth value of k; is 
equal to the value of zp. 

3. Each value of x;, beginning with z,_; and continuing to 2, is calcu- 
lated by back substitution. When all values of x have been found, 
they are given as output and the problem is complete. 


Step VII 
If, in Step IV, ¢;; = 0, go back to a;, and save a of the 7th row. Move the 
(2 + 1)st row and all succeeding rows up one position. Place the row 
previously saved into the nth position. Go back to Step III and re- 
compute a set of 4;;’s for the new 7th row. 

Figure 5.17 is a flow diagram which will result in the required row 
rearrangement. The notes indicated in the figure correspond to the 
following: 


1. The value of 7 corresponding to ¢; = 0 must be saved for future 
use. Temp is the location in which 7 is to be saved. 

2. If, at this stage, 7k is found to be equal to and f,,, = 0, no rearrange- 
ment of rows can be accomplished and the problem must stop. If 
1k is not equal to n, rows can possibly be rearranged. 

3. Beginning with aj, 7 being the row in which ¢,; was found to be 
equal to zero, each a,; is saved in a location called Save;. 

4. The (¢ + 1)st row is moved up to take the place of the 7th row. 
Each succeeding row is also moved up one row. The nth row is 
replaced by the (former) 7th row, thus completing the rearrangement. 

5. The value of zk must be increased by 1 in order that the number of 
rearrangements is available at alltimes. Variablezis reset to Temp, 
and a new set of values of ¢;; is computed. If the new ¢;; is again 
found to be zero, rows are again rearranged. Since 7k is increased 
by 1 each time a rearrangement is required, the problem will stop 
if any row is moved into the 7th row which has already been tried. 
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Figure 5.18 (pp. 115-117) is a complete flow diagram for the evalua- 
tion of simultaneous linear algebraic equations using Cholesky’s method. 


Prob. 5.10: Construct a flow diagram for the evaluation of n simulta- 
neous linear algebraic equations by use of Cramer’s rule. 

Prob. 5.11: Assuming that a relaxation table is defined, develop a 
flow diagram for the evaluation of n simultaneous linear algebraic equa- 
tions by use of relaxation methods. 
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Figure 5.18 Flow diagram for Cholesky’s method—solution of 
simultaneous linear algebraic equations. 
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Figure 5.18 (Continued) 
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Figure 5.18 (Continued) 


the inverse problem 


6.1. Introduction 


This chapter is an extension of the preceding discussion of 
simultaneous linear algebraic equations to include those sit- 
uations where the given equations are related not to a column 
vector of constants but rather to another variable. The 
more general set of equations under consideration is, then, 


[A][X] = [E][Y] (6.1) 
where [A] and [EF] are coefficient matrices and [X] and [Y] 
represent column vectors. In expanded form for a 3 X 3 
set, Eqs. (6.1) can be written as 
QiiZ1 + Ay2%2 + Aiz%3 = C11Y1 + C12Yo + C13Y3 
Goit1 + Ao2te + Go3t3 = €21Yi + C22Yo + e23Ys (6.2) 
Q31X1 + AgeXo + Azst3 = €31Y1 + €32Y2 + €33Y3 
In most cases, engineering problems and problems in 
applied science can be formulated so that each of the y’s is 
expressed explicitly in terms of one of the appropriate aj;z; 


equations. That is, [Z], the general coefficient matrix of the 
column vector [Y], is the identity matrix [J]: 


Bp) (6.3) 
118 
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For this case, the general set of equations to be considered is given as 
Eq. (6.4) or (6.5): 


[A][X] = [Z][Y] (6.4) 
Qiyt1 -F iste 4 °° * 4 Aipkn = Yi 
@o1X1 + Goote + > > > + Gonkn = Y2 
aie). 0) era Se 8 he (6.5) 
Gitte Unots 1 ~*~ si ites SS Yn 


The inverse problem is concerned with the manipulation of Eq. (6.4) 
or (6.5) so that the z’s are explicitly stated in terms of the y’s. Given 


[A][X] = [Y] (6.6) 
the equations being sought are - 
[X] = [BI[Y] = [A“1[Y] (6.7) 


where [A~'] is called A inverse. The problem is to determine the rela- 
tionship between the original a;;’s and the b,;’s, which define the inverse 
matrix. 

If the value of [Y] specified by Eq. (6.6) is substituted into the defined 
equations (6.7), it is noted that the matrix product of [A] and [A—!] is the 
identity matrix [J]. 


ia AS) = (4A) (6.8) 
or eal = TAA] = U1 (6.9) 


Extensive use of this property will be made in certain of the following 
methods for matrix inversion. 


+ 1 
oD 
6.2 Cramer’s Rule (erica 
One of the most straightforward methods for inverting a matrix is a 
direct consequence of Cramer’s rule (see Sec. 5.2). If it is assumed that 
the jth value of x is being sought, Eq. (5.17) indicates the following: 


Ba el faa 2p Sia) (6.10) 


In this equation, D is the value of the determinant of the original coeffi- 
cient matrix a,;, and D; is the value of the same determinant with the jth 
column replaced by the column of constants (in this case the y,’s). If 
expansion by cofactors along this jth column is assumed, Eq. (6. 10) can be 
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written in the summation form 


un 5d wae ~G= hae aon) (6.11) 
Aas 
or 4; = yee G = Liga ant) (6.12) 


where the A,,’s are the cofactors of the jth column elements of the original 
matrix. Clearly, Eq. (6.12) is in the general form desired; however, it 
should be noted that the 7 and 7 in Eq. (6.12) correspond to the row and 
column position in the original matrix and not to the position desired in 
the inverse set. This is perhaps better seen by an expansion of Eq. 
(6.12). 


A A A ‘A. 
a =e Vic pcg ad a ee re 
A Age A32 Ang 
te = D Yi+ D Yo + D yz + a D Yn (6.132) 
jaa Aire Aon Aan ian 
Ln DY == Dv? == D Y3 + a D Yn 
X1 An As Ag Soe oe Y1 
Xo Ay Ag Ais 2° “Ale Y2 
: 1 ase es ee aes eee ee : 
or a oy (6.136) 
Ln ‘Ada Aon Asn oes Az: Un 


In other words, the inverse matrix can be obtained by writing the trans- 
pose of the matrix composed of the reduced cofactors (cofactors divided 
by D) of the original set. In expanded form, the inverse of [A] is given 
by 

Aun Agi Az dards Ani 


eA A eee 
Veber) eo ep Ana (6.14) 
Ain Aon Aan ft Aes 


The A,,’s are the cofactors of the original a,,’s. 
To illustrate matrix inversion using this procedure, consider the fol- 
lowing two simultaneous equations: 


221 + Lo 
221 + 32 


Y1 
Y2 


(6.15) 


the inverse problem 121 


2 +1]fa]_[u 
[to ts] [2]= [2 (610 


Summarizing the various quantities required for a solution, 


>[+2 41 _|4+2 41 
a=|75 fed ie +2 13 


Au = +3 Ai, = —2 Ag = —1 Ac = +2 
The inverse is, therefore, 
+0.75 a 


Ean bee +0.50 


As a check on this result, the matrix product of [A] and [A~?] should 
equal the identity matrix: 


| =+6—-2=+4 
(6.17) 


(6.18) 


[44] 2 
EE cae bee 


+2 +3]|—0.50 +0.50 
_ [150-050 -0.50+0.50] [+1 0 (6.19) 
1.50) — 150) 983504150 | pels OF 1 


A further verification that this is the correct solution can be obtained by 
a direct-elimination solution of Eqs. (6.15). Subtracting the second of 
these equations from the first, 


—2%2 = i — Y2 
or 22 = —0.50y1 + 0.50y2 (6.20) 


Multiplying the first of Eqs. (6.15) by 3 and subtracting the second equa- 
tion from it, 
4x; = 3Y1 — Y2 


or 21 = +0.75y, — 0.25y2 (6.21) 
In matrix form, Eqs. (6.20) and (6.21) can be written as 
l=[2o8 F000] [eI (622) 
A more realistic set of equations would be 
+471+ 27 me 
+22, + 10x. + 423 = y2 (6.23) 
+ 422 + 5x3 = Ya 


+4 +2 ON ay Y1 
QO +4 +5]| 2: Y3 
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The solution is approached by first defining the values of the determinant 
and the various cofactors: 


D = +116 
Au = +34 Ais = —10 A143 = +8 
Ao a —10 Ags = +20 Ads = —16 (6.25) 


Asi = +8 Az = —16 A33 = +36 


For the problem in question, the inverse to three-decimal places is the 
matrix 
+0.294 —0.086 +0.069 
[A] = | ~0.086 +0.172 —0.138 (6.26) 
+0.069 —0.138 +0.310 


It should be noted that the original [A] was symmetric. The inverse 
matrix [A~1] is also found to be symmetric. This is a basic property of 
square matrices: Symmetrical matrices will always invert to symmetrical 
matrices. 

Since this method of inversion requires the use of Cramer’s rule for 
evaluating simultaneous equations, and since it was demonstrated in Sec. 
5.2 that the amount of work required to obtain a solution using that 
method can become prohibitive as the order of the equations increases, 
other methods of inverting matrices must be found. — 


Method *E 


6.3 Elementary Row Operators 


One of the simplest of the methods of obtaining the inverse of a matrix is 
that usually referred to as inversion by elementary row operators. Con- 
sider the set of equations 


[A][X] = [Z1[Y] (6.27) 


Assume that both sides of each of these equations is altered by either or 
all of the following elementary operations: (1) premultiplication or divi- 
sion by a nonzero constant, or (2) the interchange of any two rows, or 
(3) the addition to any row of a multiple of any other row. Moreover, 
presuppose that these various elementary operations reduce the left-hand 
side of the original equation to the identity matrix [J] times the column 
vector [X]. That is, for 


0102 -- + On[A][X] = 0162 -- - O,[Z][Y] (6.28) 
where 61, 92, . . . , 8 are the elementary row operators, if 


G09 2 "> = 6n[ A] = [I] (6.29) 
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then it is observed that 


[TX] = 6:62 - - - 6,[Z)[Y] (6.30) 
or the inverse is 


[A>] =)0103 = ~ 6,[1] (6.31) 


It therefore can be stated that a sequence of elementary row operations 
on a matrix [A] that transform it to the identity matrix [J] will similarly 
transform an original identity matrix [I] to the inverse [A-}]. 

To illustrate this, consider the set defined by Eqs. (6.24). The matrix 
in question is repeated as Eq. (6.32). 


sea eee 
(Ale Foe 210-444 (6.32) 
ee SE 


The solution using elementary row operators is summarized in Table 6.1. 
The inverse of the matrix given as Eq. (6.32) is therefore 


+0.294 —0.086 -+0.069 
[A—] = | —0.087 +0.172 —0.138 (6.33) 
0,069 7 =0.138 -+-0:310 


While certain other methods may require much less computational 
time, the elementary-row-operator method of inverting a matrix has much 
to offer. Not the least of these advantages is that it is easy to remember. 
Moreover, it is comparatively difficult to make a fundamental mistake 
when using this method. 


eee leminaton Methods Ser 


Chapter 5 was concerned with the solution of simultaneous equations 
which contained constants on the right-hand side. Therein, various 
elimination procedures were described. Extensive use of these methods 
will be made in this part of the discussion on matrix inversion. 

It should be noted at the outset that the relative influence of any one 
of the ‘right-hand constants” can be determined by setting the value of 
that particular constant equal to 1, while all other elements in the column 
vector are equated to zero. Had the value of the constant in question 
been other than the assumed 1.0, say y;, the corresponding x’s would have 
been changed a proportional amount. The procedure for examining the 
influence of each of the other y; positions would be the same. Therefore, 
to obtain the inverse of an n X n matrix by elimination methods, n 
separate simultaneous equations must be solved. In each of these a dif- 
ferent position y; is assumed equal to 1.0, and the others in that group 
are set equal to zero. Consider, for example, a 3 X 3 set. The three 


MATRIX ([NVERS ION BY ROW OPERATORS 
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Table 6.1 
ae ee ee ee 
Line Operationt Original [A] Original [J] LH 
(1) +4 +2 0 1 0 0 +7 .000 
(2) +2 +10 +4 0 1 0 +17.000 
(3) 0 +4 +5 0 0 1 +10.000 
(4) (1) + 4 +1.000/+0.500) 0 +0.250 0 0 +1.750 
(5) (2) —2- (4) +9.000} +4.000) —0.500)-+1.000 0 +13.500 
(6) (8) +4.000/+5.000)' 0 0 +1.000}-+10.000 
(499) 
(7) (5) + 9.000 +1.000}+0.444) —0.056|+0.111 0 +1.500 
(4) 
(8) (6) — 4: (7) +3 .224| +0.224|) —0.444)/-+1.000) +4.000 
(9) (8) + 3.224 +1.000}-+0.069} —0.138)+0.310}/+1.241 
Sum- 
mary: 
(4) +1.000)+0.500; 0 +0.250 0 0 +1.750 
(7) +1.000/+0.444) —0.056|+0.111 0 +1.500 
(9) +1.000} +0.069| —0.138)/+0.310) +1.241 
(1) 
(10) 2-(4) — (7) |+2.000 0 —0.444| +0.556] —0.111 0 +2.000 
(3) 
(11) |(10) + 0.444(9)}+2.000 0 0 +0.587| —0.172)/+0.138/+2.551 
(7) 
(12) (11) + 2.000 |+1.000) 0 0 +0.294| —0.086|-+0.069| +1.276 
(7) 
(13) | (7) — 0.444(9) +1.000 0 —0.087|+0.172} —0.138] +0.949 
Sum- 
mary: 
(12) +1.000 0 0 +0.294| —0.086| +0.069 
(138) 0 +1.000 0 —0.087|+0.172| —0.138 
(9) 0 0 +1.000| +0.069| —0.138)+0.310 


Final I ] | Final [A7] 


t+Numbers in parentheses refer to the contents of the designated line. 


simultaneous equations are 


Q31%1 + Aio%2 + Aisr3 = Y1 


G21X1 + Az2X2 + Aosx3 


=0 


3101 + Age%e + Ag3x3 = O 


or or 
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Possibly a more meaningful form of these equations is the following: 


Qi1 Gig Ai3] | 2 1 0 0 

Qo1 Qo2 Qo3 Lo) = 0 Y1 or = || i Y2 or = || (6) YB 

Q31 G32 33 3 0 0 il 
(6.35) 


To indicate more formally the procedure being proposed, consider the 
following: 


L1 1 0 Oly 1 0 0 
[A]} v2} =]0 1 O}}ye} =]O;yi+ H Yo + q ys (6.36) 
X3 020 “LT ys 0 0 1 


Premultiplying each of the terms by [A7}], 


L1 X1 Cr 1 0 
Pa ALA | =I) |e) = | eet = FAS 10 | ye + [A] ; Yo 
3 Z3 23 0 0 


0 
Tate ° ys (6.37) 


It should be noted, however, that 


~ first column of inverse matrix 


1 bi 0 O 
[AY H a BE 0 |- column vector corresponding to (6.38) 


second column of inverse matrix 


0 0 be O column vector corresponding to 
on ae PS ae ses (6.39) 
0 


0-0. bi, third column of inverse matrix 


> F p a column vector corresponding to 

ett 0} = 10 0 des |= (6.40) 
In other words, each of these column vectors is a column of [A~'] defined 
by the row in which the 1 appears. It should also be evident that each 
represents a solution vector of z’s for the set of simultaneous linear equa- 
tions whose column vector of constants is . 
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6.4.1 Gauss Elimination 4 Ke f 


To illustrate inversion by this method, consider the following matrix: 


+3 —3 +5 
[Al= Wal =e2ee 26 (6.41) 


+2 —Le 4a 


The solution is carried out in Table 6.2, where three sets of constants have 
been assumed. Note the similarity with Table6.1. The inverted matrix 
is 


0 +0.200 -+0.400 
[A—!] = | —0.750 —0.050 +1.150 (6.42) 
—0.250 —0.150 +0.450 
Table 6.2 
Coefficients of Constants 
Line 21 Xe X3 Y1 Y2 Y3 2H 
1 +3 —3 +5 1 0 0 
2 +1 +2 —6 0 1 0 
3 +2 —1 +3 0 0 1 
4 1 —1.000 | +1.667 | +0.333 0 0 
5 +3.000 | —7.667 | —0.333 | +1.000 0 
6 +1.000 | —0.334 | —0.666 0 +1.000 
uy 1 —2.556 | —0.111 | +0.333 0 
8 +2.222 | —0.555 | —0.333 | +1.000 
9 
10 
11 


To verify that this is in fact the inverse, the matrix product of [A] and 
[A-*] should equal the identity matrix [J]. 


om o 8 20 0 +0.200 +0.400 
+1 +2 -—6/]] —0.750 —0.050 +1.150 
+2 -—-1 +3]|—0.250 —0.150 +0.450 


+1.000 0 0 
=| 0 +1000 0 (6.43) 
0 0  +1.000 


Within the accuracy of the calculations, the identity matrix is realized. 
While conceptually the solution of n separate simultaneous equations 
is required to invert an n X n matrix by elimination methods, it should 
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be evident from the above that this does not mean that there will be n 


times the amount of work required. The entire left-hand part of the tab- 
ular solution will apply to each of the sets of solution. 


6.4.2 Cholesky’s Method —— SK) 


Table 6.3 summarizes the matrix inversion of Eq. (6.41) using Cholesky’s 


Table 6.3 


Constants Constants 


Y1 | Y2| Y3 


1 |+3/-—3/+5] 1] 0/0] 43.00 +1.67 |*+0.33 0 0 
2 |+1)+2)/—6 0] 1) 0] +1.00| +3.00 | —2.56 | —0.11 |+0.33] 0 
3 +2!—1/+3] 0} 0] 1] +2:00 +1.00 | +2.23 =n 


|<—a,;’s—>|—c’s—| 


(4ij\ti3)’S 


ina he >| 


method. (Note the similarity with Table 5.5.) The upper unit trian- 
gular set of equations is, then, 


a Bok Yi. peste U2 oe Eee 
1 — 1.0022 + 1.672, |= +0.33, 0s Po 
to — 2.00te = ——O0.11), or (= +033, or |= 0 | 
eg ee (6.44) 


The corresponding equations for the z’s in terms of the y’s are given as 
M1 = Oy: + 0.20y2 + 0.40y; 
ii oP NAY Vi A (6.45) 
x3 = —0.25y1 — 0.15y2 + 0.45y; 


The inverse is, then, 
0 +0.20 +0.40 
[A-}] = | —0.75 —0.05 +1.15 (6.46) 
—0.25 —0.15 +0.45 


Again, it should be recognized that solution for the n separate sets of 
constants does not require 7 times the amount of work. The computa- 
tional matrices [L] and [7] will be the same for all sets of constants. 
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6.5 Inversion of Matrices by Iteration or Relaxation 


Iteration methods, like relaxation methods, require independent sets of 
calculations for each assumed set of “‘right-hand-side constants.” There- 
fore, in general, n times the amount of work required to obtain one set of 
solutions (i.e., one column of the inverse) will be required to invert an 
n Xn matrix. It should be noted, however, that should a powerful 
group-relaxation-operation table be found, the labor involved in relaxa- 
tion may not be prohibitive. 


6.6 Matrix Inversion by Partitioning RERE met 


It was pointed out in Sec. 4.5 that certain matrices can be naturally par- 
titioned. For such cases inversion may be accomplished by manipula- 
tion of these smaller submatrices rather than dealing with the total 
aggregation. 

Assume that a given n X n matrix can be naturally partitioned so that 
the leading-diagonal subsets are square. 

We ara | 6.47) 
[Aoi] : [A 29] 

The orders of [Ai:] and [A22] are presumed to be p and q, such that 
p+qz=n. The inverse matrix is also presumed to be partitionable, as 
indicated in Eq. (6.48). 


[Bi] | a (6.48) 


[A-] = | aes aed 

[Boi] : [Boo] 

Here, it is further specified that [Bi:1] and [Bz2] are also, respectively, of 
order p and gq. 


From Hq. (6.9), the condition for matrix inversion is 


[A][4—] = [J] (6.49) 
Substituting the values defined by Eqs. (6.47) and (6.48) in Eq. (6.49) 
gives 
[Au] eel Fac " Fat 
ee (aul) LiBal told ~ Lol tH le 


Multiplying the two matrices in the left-hand member and equating the 
result to the right-hand member yields 


[AiJ[Bii] + [Ai2][Ba] = [7] (6.51) 
[Aii][Bi2] + [4A12][Boo] = [0] (6.52) 
[AsiJ[Bu] + [Ave][Boi] = [0] (6.53) 


[A2e1][Bie] ae [A 22][B22] a (Z] (6.54) 
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Multiplying Eq. (6.53) by [A12][A 2274], 
[Ar][A oo )[Aoi][Burl + [Ar2][A22 [A o0][Bai] = 0 (6.55) 
Noting that [4o.—][4o0] = [J], 
[Ar2][A ee [A oi][Bui] + [Ai][Bai] = 0 (6.56) 
Subtracting Eq. (6.56) from (6.51) yields 
[An][Bu] — [Ar2][A227 [4 o][Bui] = [J] 


or [Au] = [A y2][A oo 4[A 21] |[Bai] =" {7} (6.57) 
Solving Eq. (6.57) for [Bui], and noting that in general if [M][Z] = [J], 
then [Z] = [M-"] = [M}-}, give NEED iw VERSES OF 
[Bu] = [[Au] = [AvI[Aa All) (6.58) A,, z AL 
From Eq. (6.53), oT 
[Bei] = [—[Ae2][Aei][Bul] (6.59) 


In a similar manner, Eqs. (6.52) and (6.54) will yield Eqs. (6.60) and 
(6.61). 


[B22] = [[A 22] a [Asi][A 1 J[A18]|-? 
[Bi] = [| —[Air [A 12][Boo] | 


The inversion of an n X n matrix by partitioning thus reduces to the 
inversion of four matrices. Two of these will be of order p. The other 
two will be of order g. Several matrix multiplications and additions are 
also required. 

To illustrate inversion by this method, consider the matrix defined by 


Lee ET 0RR0 
Oe i 0 [Au] [Aid] 


AJ=|-1 O 41) O 41] =] oe ; 
2 al aes (reniel ee) 


(6.60) 


and (6.61) 


Ou (Oier Os 0 <b 2 


The required inversions of the original submatrices are as given in Eqs. 
(6.63). 


+0.50 0 —0.50 
[Ansel = 0 +0.50 0 [Ay] = (eas 0 


HO0S0m 08 | 0.50 ee ea 
(6.63) 


Considering Eq. (6.58), 


0 0 0 0 0 

+0.50 0 0 0 0 
[A12][Aoe [Aa] = | +1 o|| || J=[2 09] 
12||A 22 21 ; 3 0 +0.50};0 0 0 000 


(6.64) 
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Therefore, 
+0.50 0 —0.50 
[Bir] = [Air?] = 0 +0.50 0 (6.65) 


For Eq. (6.59), 


+0.50 . O —0.50 
[Aso ][A oi] [Bui] = bea anges 5 v | 0 +0.50 0 | 


or [Bai] = [i 0 ‘1 (6.67) 
Consider Eq. (6.60). 


000 +0.50 0 —0.50 0 0 
[AnlldLtsal = | 9 : 0 ene ae ea dit eat 
+0.50 0 +0.50 Oo +1 
0 0 
=i; | (6.68) 
0.50 0 
or [Boe] = [Aa2?] = Be ee (6.69) 


Equation (6.61) requires that 


[A wu J[A 12][B29] 
[0 0 —0.50 


0 +0.50 0 


ms ape 0 
0 +0.50 


+0.50 0 +0.50 We 
0 —0.25 
= | +0.25 0 (6.70) 
0 +0.25 
0 +0.25 
or [Bio] = | —0.25 0 (6.71) 
0 —0.25 
The inverted matrix is, therefore, 
+050 O -050  O +40.25 
0 +0.50 0 {| —0.25 0 
[Ana] | tO 50, aD eect UOT) 2 Oe ee 0.2 5) a car) 
0 0 0 - +050 0 


0 0 Oj 0 40.50 
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To verify that this is in fact the inverse, the matrix product of [A] and 
[A—1] is shown in Eq. (6.73). 


tae DL ciel 0 0 |} +0.50 0 — 0.50 0 +0.25 
Om 0 +1 0 0 +0.50 0 —0.25 0 
—A Oe oe O +11)] +0.50 0 +0.50 0 —0.25 
0 0 OS 4 0 0 0 0 +0.50 0 
0 


0 0 0 Os -F2 0 0 0 +0.50 
abl 0 0 0 0 
Oss 0 0 0 
= 0 Ot 0 0} (6.73) 
0 0 seh 0 
0 0 0 Os +1 


——, 
NA a By Cramer’s rule, elementary row operators, Gauss 
elimination, and Cholesky’s method, invert the following matrices: 


Ciel 0. —1 CO ee 
0 +1 0 0 +1 +3 
+1 OO +41 Om ar0.ac ri 


(c) [+0.25 +0.30 —0.10 (d) [—0.294 +0.086 —0.069 
+0.20 —0.50 +0.15 +0.086 —0.172 +1.138 
[-0.15 +0.35 +0.60 —0.069 +0.138 —0.310 


<_ Prob. 62: For the following 3 X 3 set of equations relating x’s and 
y’s, determine by Cholesky’s method the y’s as explicit functions of the 
x’s. 


— 


3821 + 2%2 + 23 = lyr + 2y2 
241 + 522 — 223 = 2Yy1 sm OE) oie ly: 
ee 12, = 22 + 4x3 a ly2 + 5Ys 


rob. 6.3 by partitioning, determine the inverse of the following 
matrices: 


(a) te oy kl) 
2210 

[ANS teo Po 
000 3 

(b) EO MORON O 
Ont 204k0. 1.20 
Oda Utell Oyn0 ake! 

be TE Cae 
TPP) an eee 
6-001-10.0, 2 
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(Prob. 6.4: Given the symmetrical 4 X 4 tridiagonal system shown, 
determine-the inverse of the set by partitioning. 


1, aoe 6 
Pe, AY 
OZ Lee 
0. OF 1 3 


Prob. 6.5: It is assumed that a matrix can be partitioned so that 
[Ay.] = 0. In terms of the submatrices [A11], [A 21], and [A22], what will 
be the elements of the inverse of such a system?- Using these results, 
determine the inverse of the following matrix: 


Prob. 6.6: If in Prob. 6.5 the matrix had been ‘‘bordered’”’ [see Eq. 
(4.56)], as shown in the following set, 


108 1 0F80 
17:2 0) 040 
19° il 0eK0 
1, OjrOn, 0 
C116 O00 02 


would the derived equations also apply, and if so, would any computa- 
tional advantage be realized? 

Prob. 6.7: It is assumed that an arbitrary lower triangular matrix will 
invert to another lower triangular matrix. For a3 X 3 system, develop 
the equation defining the elements of the inverse. Are there any special 
limitations on this set of equations? 


Prob. 6.8: In Example 4.4, it was presumed that voltages EH; and E, 
were specified and that values of the unknown currents J, I2, I3, and I4 
were desired. Assuming that all the resistances are equal, i.e., 


Ri= ho2= Rs; Ri =R; =k, =R 
and that the currents are known, determine the expressions for E; and EF, 
in terms of the J’s and R. 
Prob. 6.9: Example 4.8 was concerned with the transformation of a 
generalized force-couple system in rectangular coordinates 2, y, and z to 


a parallel axis system , g, andZ. The equations defining the transforma- 
tion are given as (4.101). Obtain the inverse of this system; i.e., define 
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V; as an explicit function of V;. From geometrical considerations and 
Eqs. (4.101), verify these new equations. 


6.7 Computer Solutions 


' More often than not, engineering and other problems in applied science 
are “directionally formulated’’; that is, one of the quantities in question 
is more easily defined in terms of the remaining variables than vice versa. 
For example, in structural analysis, deflections are readily described in 
terms of geometrical properties of the various members and the imposed 
loading. In electric circuits, voltages can easily be defined in terms of 
resistances and currents. While this is true, the reverse (or inverse) 
formulation is frequently the more desirable. This is especially so when 
design problems are considered. 

Of the several methods for the inversion of matrices that have been 
discussed, perhaps the most adaptable to computer solutions are those 
based on elimination methods. The case that has been chosen for illus- 
tration is that of repeated application of Cholesky’s method. 

It will be recalled from Sec. 6.4.2 and Table 6.3 that Cholesky’s 
method, extended to the inverse problem, requires that the coefficient 
matrix [A] be augmented by an identity matrix [J] of order n. By 
operating on each column of this identity set as if it were a separate 
column vector of constants, that is, by considering 7 different sets of 
simultaneous equations, the n different columns of the inverse [A—"] can 
be established. 

It is necessary, in writing the computer program, to ensure that each 
specific column of the identity matrix in the augmented set corresponds 
to its respective column in the inverse. To facilitate this, a special 
counter denoted as m will be used to define the columns of the inverse 
matrix. 

A flow diagram that will invert a matrix by use of Cholesky’s method 
is shown in Fig. 6.1. It will be noted that use is made of the routine for 
the solution of simultaneous equations that was presented earlier as 
Fig. 5.18. It has been assumed that a simultaneous-equation program 
is available for use as a subroutine. 

The notes in Fig. 6.1 refer to the following: 


1. As stated above, m is a counter which defines the column of the 
inverse matrix currently being calculated. It will vary from 1 to n. 
2. At this stage, beginning with 7 = 1, a test is required to determine 
whether 7 = m. If such is found to be the case, the corresponding 
element in [C] or c; must be set equal to 1. All other c,’s must equal 
zero. Inthis manner the required column matrices c; are generated. 
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Notes 


at] [a7 


© 


Substitute cy for a; 44 in | 
Cholesky's method (Fig. 5.18) 
and solve for x; 


raft 


Figure 6.1 Flow diagram for matrix inversion—Cholesky’s 
method. ; 


3. The actual flow diagram for the solution of the simultaneous equa- 
tions is presumed known. (Entry into that flow diagram is made 
at note 2in Fig. 5.11.) Exit from the simultaneous-equation rou- 
tine to this program would be just prior to the stage where the 2; 
are to be put out (Fig. 5.16). 


4. Ann X n matrix is formed, the columns of which are the values of 
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the z,’s just determined. When all n columns have been established, 


that is, when m = n, the values of Lim are read out as the elements 
of the inverse [A-?]. 


Prob. 6.10: Construct a flow diagram for the inversion, by partition- 


ing, of an nth-order set using Eqs. (6.58) to (6.61). Introduce checks to 
ensure that the matrix operations can be realized. 


eigenvalue problems 


7.1 Introduction 


In all the preceding work on matrices and simultaneous equa- 
tions, it has been assumed that the coefficient matrix is non- 
singular. That is, the value of the determinant of the coeffi- 
cients is other than zero. It should also have been noted that 
in each case the elements of the various assumed matrices 
were constants. Another observation that could have been 
made is that the equations considered were nonhomogeneous. 

Exgenvalue problems are concerned specifically with the solu- 
tion of sets of homogeneous equations (1.e., all elements of the 
column vector of constants equal zero). For illustration of 
the general problem, consider the set of equations 


Puvti - Pists + “>= - Pirtm— 0 
P21%1 = P22Xe os Pantn = 0 

(7.1) 
Pnivti ain Pn2X2 sh ooh =F Pnrntn = 0 


It will not be assumed necessarily that any or all of the coef- 
ficient elements p;; are constants. 
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One obvious, though trivial, set of solutions to the simultaneous equa- 
tions (7.1) is 
igh == Hey Sy Oo OO a apy ah) (2:2) 


There are, however, other possibilities. From Eq. (5.17) (Cramer’s rule), 
x; can be defined as 


Li = 


de 
: (7.3) 
where D is the value of the determinant of the coefficient matrix [p;,] and 
D; is the value of the same determinant with the jth column replaced by 
the column of zeros (the vector of constants). Evaluating D,; by expan- 
sion about the jth column (now replaced by zeros), 


D; =0 (7.4) 
Therefore, for x; to have a value other than zero, a necessary and suffi- 


cient condition for solution is that the determinant of the coefficient 
matrix equal zero. 


D=0 (7.5): 
fori, U5 foe SO 

or Dee ee ee en (7.6) 
Pni Pn2 Pn3 Pnn 


It will now be assumed that the coefficient matrix [P] is actually the 
difference between two other matrices [A] and [B]. Moreover, it will 


further be presumed that [B] is a coefficient matrix [B] multiplied by an 


unknown scalar \. [P], then, is defined as 
[ey 14l— x18} (7.7) 
or Diy = Oa dba; (9) = 1} Da A eOL TO. n) (7.8) 


and the set of simultaneous equations under consideration is given as 
(7,9) 01.710), 


[[A] — A[B]][X] = 0 (7.9) 
(11 = Nbi1) 21 + (d12 = Abie) x2 qe oe (Ain a Nbin)&n = 0 
(dex a Nbo1) 21 + (d22 = Abo2) x2 Se ON 3 (don a Nbon) En = 0 (7.10) 


ae state 2 6 ialie tie? 0 a Mea ke? Ce. os) FeMined fol Ms ysl eie) 14,0 ue) net) 0.8 ye" 16) Xe.) ceQege ee [enero 18 


(Qn1 a NOn1) 21 =i (Ano > NOn2) Xe Sie hs + (an = NOanlen = 0 
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Here again the necessary condition for a solution is that the determinant 
of the coefficient matrix equal zero. That is, 


|A — »B| = 0 (#11) 
@11 — ADir iz — ADi2 °° * Gin — Din 

ar G21 — bei G22 — Aber °° ° Bos = ae =i} (7.12) 
Gni — ABn1 -Gn2 — ADn2 °° * Onn — ADnn 


By expanding Eqs. (7.11) or (7.12), there is obtained an nth-degree 
polynomial in the unknown )’s. This can be written in the form 


Mt GAT + gad? s+ tgedt+q,=0 (7.18) 


The n roots of this polynomial are known as the eigenvalues. They corre- 
spond to those particular \ values which allow the determinant of the 
coefficient matrix to equal zero. It should be noted, however, that these 
eigenvalues may not be all real or distinct. 

Corresponding to each of the n eigenvalues is an ezgenvector, that is, the 
column vector of x values. It should be noted that since a solution has 
essentially been forced (A chosen so that D = 0), no one unique set of x’s 
can be found. The ratio between the various x values can, however, be 
established, as will be proved in the following paragraph. 

From Eq. (7.9), 

[A][X] = Ai[B][X] (7.14) 


which assumes that [X] is an eigenvector corresponding to the eigenvalue 
1. Assume now that [aX] is also an eigenvector, where a is a nonzero 
scalar. 


[A ][aX] = a[A][X] = ad, [B][X] = Ai[B] [aX] (7.15) 


That is, eigenvectors can be determined only to a proportionality. 

In many real problems the coefficient matrix [B] is the identity matrix 
[I]. When such is the case, the resulting polynomial in } is often referred 
to as the characteristic equation or secular equation. The corresponding 
eigenvalues are frequently denoted as proper values. 

Thus far it has been presumed that the d values are scalar multipliers 
of the coefficient matrix [B]. This need not be the case. \ may be con- 
tained directly in matrix [B] as the argument of a trigonometric, loga- 
rithmic, Bessel, or any other function. Consider for illustration the 
following equations, which define a rather elementary buckling situation 
(see Example 4.1). 


0 1 0 1{la 0 
sind cosd 1 1]|as| |0 (7.16) 
sind cosr O O]| ax 0 
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The two matrices [A] and [B] are 


Cereal 
001 0 
[A] = OF omit (7.17) 
OO 0iR0 
0 0 0 0 
eee a0 
Lele ee sind cosd 0 O Cia 
snd cosr 0 0 
where [A — B][X] = [0] (7.19) 


7.2 Examples of Eigenvalue Problems 


Many engineering problems require the determination of eigenvalues and 
their corresponding eigenvectors. Three very general areas in which this 
becomes the fundamental consideration are as follows: (1) natural fre- 
quencies of vibrations, (2) buckling, and (8) resonance in acoustical and 
electrical circuits. A wide range of physical situations would be encom- 
passed in each of these. For example, consider just a few of those associ- 
ated with the first area: (1) the natural vibration of systems of masses 
and springs, (2) the flutter of airplane wings, (3) the vibration of mem- 
branes, (4) the oscillation of suspension bridges, (5) the torsional vibra- 
tion of multicylinder engines, and (6) structural response to earthquakes. 


Example 7.1 Asa first example, consider the set of two simultaneous 
equations 


41 + 2x. = AX 1-—~2z 2 V1 0 
= 7.20 
aps 4 v2 = AL2 oe | 2 1- | es la ( ) 
Since these equations are homogeneous, the necessary condition for a 


solution is that the value of the determinant of the coefficient matrix be 
zero: 


yet 0 (7.21) 
Expanding Eq. (7.21) gives the characteristic polynomial equation as 
2— 21-3 =0 or (A+ 1) — 3) =0 (W-22) 
The eigenvalues are therefore 
AM = +3 and = —l1 (7.23) 


To determine the eigenvectors for these roots (eigenvalues) it is neces- 
sary to assume the value of one of the x’s; for example, select x1 = +1.0 
ford; = +3. From the first of equations (7.20), the x2 value is found to 
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be 
to = +1.0 (7.24) 


As a check, the same values of 21 = 1.0 and \1 = +8 can be substituted 
in the second of the given equations. Here, again, it is found that 
ze = +1.0. Therefore the first eigenvalue and corresponding eigenvector 
are 


seh ee aw) 
M=+3 £[X]= Bee (7220) 
For the negative root \2 = —1, the second eigenvalue, the solution is 
a __ (bo 
Ae =.-1 [X= Esa (7.26) 


Example 7.2 


nN +2 +2 L1 0 
0 Zee —! tg} = |0 (7.27) 
=e == 7s +3 —r4-423 0 


Equating the determinant of the coefficient matrix to zero gives 


—2—- 2 —1 +2 ie 
(—1 —)) = a eae lee (7.28) 
or N= 0 = AR Oe) (7.29) 
The eigenvalues are therefore \1 = 0, Ax = +1.0, and A; = —1.0. -The 


eigenvectors can be determined in the same manner as in Example 7.1. 
The solutions are summarized in Eqs. (7.30) to (7.32): 


+1.0 

1 = 0 [x] =| -0.5 (7.30) 
10 
a i0 +0.67 

m= +10 [X] =] -05 or —0.33 (7.31) 
+1.5 +1.00 
=15 +1.00 

re Sikh = sbah oa ccing or —0.67 (7.32) 
= +0.67 


It should be noted that two eigenvectors have been listed for each of the 
last two eigenvalues. The first of these corresponds to the assumed +1.0 
as one of the component values (a; in the second and 2» in the third). 
The second sets of eigenvectors were obtained by normalizing with respect 
to the largest component. 


Example 7.3 The problem under consideration is the determination 
of the natural frequencies of vibration (i.e., eigenvalues) of the two-mass 
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i, i, 


2k 


A/\\s 


Figure 7.1 


spring-connected system shown in Fig. 7.1. The corresponding eigen- 
vectors are also desired. If a disturbance from the neutral position is 
assumed, the free-body diagrams for each of the masses (including 
d’Alembert forces) are those shown in Fig. 7.2. It has been presumed 
that we > uw: and that friction can be neglected. 

From equilibrium of horizontal forces, the equations given as (7.33) 
are obtained. 


—kui + k(ue — wi) = mia 
(7.33) 
—kK(uez — U1) oa 2k(u2) = Mu 


For the natural frequencies, the displacement quantities (wi and we) can 
be related to the amplitudes of vibration (ai; and a2), the natural (or cir- 
cular) frequencies (w), and a phase angle (¢) according to Eqs. (7.34). 


ui = a, sin (wt + ¢) 

Uz = a2 sin (wt + ¢) 
Substitution of Eqs. (7.34) in Eqs. (7.33) yields 
—ka, sin (wt + ¢) + k(a2 — ay) sin (wt + ¢) 
—k(az — ay) sin (wt + ¢) —2kae sin (wt + ¢) = —maw? sin (wt + ¢) 


(7.34) 


I 


—mMayw* sin (wt + ¢) 


—2+2X t= 0 
te iggy are (7.35) 
ax(1) + a2o(—3 + A) = 0 
2 
where A= a= (7.36) 


The necessary (and sufficient) condition for Eqs. (7.35) to have other 
than a trivial solution is that the value of the determinant of the coeffi- 


Figure 7.2 
uy | Ws 
k (uy) k (U2 —U4) k(Uo—U4) 2k(Uo) 
| ee LIL prc eM 9--— 


142 modern methods of engineering computation 


cient matrix be zero. That is, 


— 2—x -—l 
pious mee -| =i faeo Cy 

The characteristic polynomial equation is therefore 
’w—5.+5=0 (7.38) 

The eigenvalues, which would be the roots of Eq. (7.38), are 
5 5 

M1 = eee = +1.382 

(7.39) 


a/5 


5 5 
aa +3 53.618 
Misael est aeapetiemeee 


The solution can, therefore, be summarized as in (7.40). 
Ar = +1.382 Ae = +3.618 


13824 018 2 
Oy aa} eS2s we = 4/3.618 — 
m m 
Oy) cutee (e000 ere back rs 
Be: ia € Meera | orale This | i bourne eo 


A qualitative picture of what these mean physically is shown in Fig. 7.3. 


Example 7.4 <A flagpole type of uniform cross-section column is 
subjected to a concentrated vertical force at its uppermost point, as shown 
in Fig. 7.4. It is assumed that the vertical force will remain vertical. 
The differential equation governing the solution of this problem is 


dty d*y 
ee, a : 
al x? 0 (7.41) 
P 
h ' k? = — AS 
where ai (7.42) 


The solution to this homogeneous differential equation is 
y = Ci sin kx + C. cos kx + C3 = + C4, (7.43) 


The boundary conditions that must be satisfied are the following: 
Atz=0: y=0 

(0)C1 = (1)C2 + (0)C3; + (1)Cs = 0 
Atc = 034 = 0) 


HC + OG + (7) Cr+ @C.=0 


7.44 
Atz=L: y’ =0 Shee 


(—k? sin KL)C, + (—k? coskL)C2+ (0)C; + (0)Cs = 0 
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K 
~t+1.000H ~+ 0.618 ~—0.618 ~+1.000 

One 
period 

~+1.000 

F | ~+0.618 ~+ 0.618 ~+1.000 
Time 
pe Uy i Ls Up 


Be eh, 


Both masses move Masses move in 
in same direction opposition 


Figure 7.3 
Bt ees 7! 4. hy) = 0 
k 
Ve + “OC, + @) Cr+ (OC. =0 


By defining kL = i, Eqs. (7.44) can be written as 


0 it OP bes 0 
d OMRON CMe 150 
sin r= cosA 0 O}1 C3) | 0 ee) 
0 0 Pee Otic, 0 
Figure 7.4 


ale 
| 
\ 
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For a nontrivial solution to exist, the determinant of the coefficient matrix 
must equal zero. This gives, on expansion, 


A cos A = 0 (7.46) 
But» = 0 will also result in a trivial solution (that is, P = 0). Therefore, 
cos A = 0 (7.47) 
or » = On+ 1) = (00 eee) (7.48) 


The smallest eigenvalue (that is, the first possibility of buckling) occurs 
for the case where n = 0. 


: Tv IP 
=e = yh = A\—— 7.49 
M1 5 iL EI L ( ) 
2HT 
or Py = aaa (7.49a) 


From the third of Eqs. (7.45), if it is noted from Eq. (7.47) that 
cos\ = 0,C, = 0. From the fourth of these equations, C; = 0. There- 
fore, from the first equation, 


C, = —C, (7.50) 
or the deflection equation is 


i 1 — cosXr (=) (7.81) 


For the case \1 = 0.57, the deflection equation is 


Pe. E = cos* (7) | (7.52) 


The next larger eigenvalue corresponds to the case where n [in Eq. 
(7.48)] equals 1. That is, 


3 On? HI 
Ay = a or P, = a 


(7.52a) 


The corresponding deflection equation is 


hac E sf cos = (2) (7.52b) 


The deflection configurations corresponding to these buckling solutions 
[Eqs. (7.52) and (7.52b)] are shown in Fig. 7.5. Here again, it should be 
noted that only relative values are known. 
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de 


Figure 7.5 


7.3 Determination of Eigenvalues by Iteration 


Frequently, engineering problems require the determination of either the 
greatest or the least eigenvalue. When such is the case, iteration may be 
a convenient method of solution. 

The general equation under consideration is (7.9), which is here repro- 
duced as (7.53): 


[A][X] = A[BILX] (7.53) 
This could have been equally well written as 


[B~N[A]LX] = AB I[BILX] 


[B~'A][X] = d[X] (7.54) 
or [A—][A][X] = [A ][B][X] 
[X] = \[A-!B][X] (7.55) 


By noting that d is a scalar quantity, Eq. (7.55) can also be written as 
1 
[A—!B][X] = A [X] (7,000) 


In essence, the iterative procedure for determining eigenvalues is based 
on the condition that should a trial vector ;[X] be assumed, an approxi- 
mate eigenvalue and a second trial eigenvector ,4:[X] can be determined 
from either Eq. (7.54) or Eq. (7.55a). It should be noted, however, that, 
as is true in all iteration methods, convergence is the real problem at issue. 
A strong diagonal set is a sufficient condition. 

For illustration of the general method of solution under discussion, con- 
sider the vibration problem given as Example 7.3. In the matrix form of 
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(7.54), Eq. (7.35) can be written as 


Ee ie a — E | a (7.56) 


The equation corresponding to (7.54) is 


+2 —-l1 ay ay 
= i. 
a la eric 
In the terms of Eq. (7.55a), this same set can be written as 
+0.20 +0.40 ] | ae r | ae 


Equation (7.57) will first be considered, and as a starting point for the 
iteration procedure it will be assumed that the eigenvector is 


AX] = I = Fee (7.59) 


In general, convergence will be facilitated by choosing as an initial trial 
eigenvector the value +1.0 in the position corresponding to the strongest 
element in the [A] matrix, while the others are kept zero. 

Carrying out the matrix multiplication on the left-hand side, 


+2 —-1 0 by cat BD 
‘ie A eal os pee SB) 
If this column vector is now altered by removing the scalar quantity.+3 


(that is, by normalizing with respect to the a2 element), it is observed 
that the form is exactly that of the right-hand side of Eq. (7.57). That is, 


00333) 8 ay 
+s[ 28] = af cn 
At this stage, then, a first approximation to one of the eigenvalues is 


1i\ = +3. The next trial value for the column vector of a’s would be 
ad, = —0.33 and ag = +1.00. A few of the trials are carried out below. 


Leen Se ae) eal 
| cies | leeon l= legen Ube 20 etre gal ome oa 
| ty ora [pied = [ete soo sien ee erior ema el 
Pree a ae of 


Table 7.1 summarizes the first nine iterations. It should be noted that 
the iteration process does converge to the correct larger eigenvalue with 
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corresponding components for the eigenvector. [See Eqs. (7.52).] In 
general, the iteration procedure using Eq. (7.54), if it converges, will yield 
the largest eigenvalue. 


Table 7.1 

ee er 
Itera- 

ona 2, 3 4 5 6 i 8 9 


dA = +3.00 +3.33 +3.50 +3.572 +3.601 +3.612 +3.614 +3.617 +3.618 


ay 0.33 —0.50 —0.572 —0.601 —0.612 —0.614 —0.617 —0.618 —0.618 
a +1.00 +1.00 +1.000 +1.000 +1.000 +1.000 +1.000 +1.000 +1.000 


Consider now the same problem, but expressed in the form of Eq. (7.58). 
As an initial trial assume a; = +1.0 and az = 0. 


ae ee peal x pee a 00 | +1] il [2 | 
+0.20 +0.40 | ae OO ee 10.83). a as 


The solution is summarized in Table 7.2. Here again the iteration con- 
verges. However, this time it is to the lower eigenvalue. In general, 
the rteration procedure using Eq. (7.56a), if it converges, will yield the small- 
est ergenvalue. 


Table 7.2 


Itera- : 
tions: e 


1/ +0.60 +0.667 +0.700 +0.714 +0.720 +0.722 +0.723 +0.723 
» +1.667 +1.500 +1.429 +1.400 +1.390 41.385 +1.382 +1.382 


a +1.00 +1.000 +1.000 +1.000 +1.000 +1.000 +1.000 +1.000 
a2 +0.33 +0.500 +0.572 +0.600 +0.612 +0.615 +0.617 +0.618 


It should here be noted that iterative methods can be developed for deter- 
mining intermediate eigenvalues. However, it is considered that these 
are beyond the scope of this introductory work. 


7.4 Modification of the Coefficient Matrix |B] 


Both in the exact expansion of the determinant and in the iterative 
method, a solution can be facilitated if [B] can be reduced to the identity 
matrix or at least to that case where the leading diagonal elements are 
all 1. 
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When the [B] matrix is a diagonal matrix of the form 


ii) Ohiby 8 its v1 bi 0 she tes 0 U1 
Qo1 Goo °° * Gon X2 0 Dont “Pe? 0 Xe 
=} 

Qni1 Qne2 °° * Qnn Xn 0 0 ee ban Xn 


a direct division of each equation by the 6b; term»will give 


Coplay 2 S fadiys, v1 U1 
Copy) apy 0 OY Le Lo 
= NJ] 
Qn1 OAn2 Ann In Xn 
aij 
where a5 
bi 


The determinant to be solved is 


Oye N Q12 ie Qin 
Q21 @22 — Q2n =i 
Qnl An2 Se One aN 


(7.62) 


(7.63) 


(7.64) 


(7.65) 


This type of problem frequently occurs in lumped-parameter vibration 


problems. 


Prob. 7.1: Determine the eigenvalues and eigenvectors for the follow- 


ing set of equations: 


—12 —24 —87)} a 1 
0 — 24 —32 V2) = AL] iD) 
AR Aa G ag Xe 


Figure 7.6 


Frictionless 


D pin connection 
Ay yr 
Fring. 
ionles. f 
Ib/e 


Frictionless 
PIN Connection 
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rh Ly=L | Lo=L 
{ise a aa 


C2=C 


Figure 7.7 


Prob. 7.2: By iteration, determine the largest and smallest eigenvalues 
for the following set of equations. 


+4 +2 0 V1 V1 
ILO ead lee een Te 
O +4 +5]] 23 X3 


Prob. 7.3: Two masses m, and m2, assumed to be resting on a friction- 
less table, are connected as shown in Fig. 7.6. The links connecting 
points A and B and points C and D are assumed to be inextensible. The 
spring between B and C is presumed to have a stiffness of ki. Write in 
matrix form the equations governing the determination of the natural 
frequencies of vibration of the system. 

Prob. 7.4: A column of uniform cross section, subjected to a vertical 
load at its uppermost point, has the following boundary conditions: 


alt pie aaa Ws y=0 and ae ING) 
At.a = L: y =0 and y =0 


Using Eq. (7.43), determine the buckling loads. 
Prob. 7.5: Assuming that the currents at resonance can be expressed 


as f 
=" Sin (wt + ¢) 2 = a2 sin (wt + ¢) 


and the governing differential equations are of the form 


1,0 


‘Te = 0 (even 12) 


where J; is the current in L;, and J; is the current in C;, determine the 
resonant frequencies for the circuit given in Fig. 7.7. 

Prob. 7.6: Two masses m and m: are suspended from springs as 
shown in Fig. 7.8. The springs have stiffnesses of k; and ky. What is 
the characteristic polynomial governing the eigenvalue solution to this 
problem? 

Prob. 7.7: Determine the characteristic equations for the following 
sets of equations: 
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Figure 7.8 
(a) | +2 —-1 0 O}] x1 +1 0 0 0 || a1 
—] 42 -2 Oo ees Ea Q +2 0 0 || xe 
Q —2 +4 =4 |\a5) ~~ 0 Q +2 0 || xs 
0 QO -—4 +6 || 2% 0 0 On “orate 
(b) | +2 —-1 0 On Ge ame! 0 OQ o-—1 bigre 
—] +2 -1 0 || xe ii 0 42 +1 0 || xe 
O--—1 44 —2/| a3] 0 +1 +2 -+1 || 2s 
0 0 -—2 +4) x4 —1 QO +1 +3 }| 24 


7.9 Computer Solutions 


From the foregoing examples and problems, it should be evident that for 
larger-order systems, the numerical work associated with the direct alge- 
braic solution of the normal eigenvalue problem [Eq. (7.11)] can become 
prohibitive. Iterative methods, on the other hand, can be numerically 
efficient for these types of problems provided only the lowest or highest 
eigenvalues are desired. For the determination of intermediate eigen- 
values using iteration, it is necessary to develop methods for the removal 
from any assumed, or computed, eigenvector that part which corresponds 
to the higher (or lower) eigenvalues. These procedures can become quite 
complex. It should also be evident that many ‘‘real problems” contain 
the eigenvalue as a function of trigonometric or other functions. The 
algebraic methods that were discussed earlier in this chapter are not 
readily applied to these situations. For these and other reasons, many 
eigenvalue problems are evaluated by direct, systematic, trial-and-error 
procedures. It is this method that has been chosen for illustration. 

The general problem type that will be considered is defined by the 
matrix equations (7.66) and (7.67). 


[[A] — A[B]][X] = 0 (7.66) 
[[A] — [BA)]][X] = 0 (7.67) 
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In Eq. (7.67), the symbol [B(A)] has been used to indicate that the matrix 
[B] can be any function of the unknown X. 

It was demonstrated earlier in this chapter that eigenvalue solutions 
correspond to those particular \’s for which 


a= A= NB 0 (7.68) 
or D = |A — BQ)| = 0 (7.69) 


In this section, the general computational method that will be used to 
obtain computer solutions is as follows: A trial value of } is first selected. 
With this, the determinant D, defined by either Eq. (7.68) or Eq. (7.69), 
is evaluated. Should D = 0, the trial value of \ corresponds to one of the 
eigenvalues. Should D be other than zero, a new d is selected and D is 
again determined. If D is again not equal to zero, a comparison is made 
with the results of the previous trials and, based on these comparisons, 
a new ) is selected for trial. It should be noted that such a procedure can 
be made to give all the real eigenvalues contained within a given pre- 
selected range of }’s. 

Seldom will a \ be found for which the value of the determinant is 
exactly zero. More often, a d will be found for which D will be “‘suffi- 
ciently close to zero.” It is necessary, then, to define at the outset the 
magnitude of the error to zero in the absolute value of the determinant 
that will be allowed. It will also be necessary, for automatic computa- 
tion, to define the original increments in the trial values of the eigenvalues. 


Figure 7.9 


Read in all problem data and initialize all | 
problem variables, 
| = 


From the elements of the coefficient matrices 
[A] and [8] and a trial value for d, 
construct a new matrix [0] from the equation 


[0] = [4] - » [6] 


Evaluate the determinant of the new matrix [D] 
tee. 2) and determine its absolute value. 


Compare the computed absolute value of D with 
the error term &.|f it is sufficiently small 

(i.e., 0 < &),the current value of \ is acceptable. 
\f, however, D> & , the trial value of \ must be 
increased and the process repeated. 
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Step I 


Figure 7.10 


In broad terms, a flow diagram that will determine all the real eigen- 
values contained within a preselected range of )’s is given as Fig. 7.9. 
The detailed diagrams for each of the parts are shown in Figs. 7.10 to 
7.12. A complete flow diagram is contained in Fig. 7.13. 


Srrep I 
Read in all problem data and initialize all problem variables. 
A detailed flow diagram that will carry out these instructions is given 


in Fig. 7.10. The notes indicated in Fig. 7.10 have the following 
significance: 


1. The variables shown in this box represent, in the order given, the 
following: 


V = any trial value for an eigenvalue (A) 
T = maximum possible value of (A) that is of interest 
Ax = original increment(s) in V 
EK = error term which defines acceptable deviation from zero of 
absolute value of determinant 
2. Since an incremental procedure is to be used, Check has been set 
aside to hold the last calculated value of the determinant. At each 
step, the new value of the determinant is compared with Check. 
When Check < E, an acceptable eigenvalue has been established. 
3. Val is used to store the value of V. 
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4. s is a variable which defines the current increment to be taken. 
As a solution is approached, s will become successively smaller. 


5. k is a fixed-point variable which defines the eigenvalue that has 
been found. 


Step II 
From the elements of the coefficient matrices [A] and [B], and a trial value 
for \, construct a new matrix [D] from the equation 


[D] = [A] — A[B] 


Evaluate the determinant of this new matrix [D], that is, D, and determine 
its absolute value. 
The following are noted in Fig. 7.11: 


1. Point @) is the position through which control returns to evaluate 
any trial determinant. 

2. To eliminate the problem of destructive read-in, (bm),; and d,; are 
set up as new variables. 


3. It is presumed that a determinant-evaluation subroutine is available 
(see Fig. 3.7). 


Figure 7.11 
Notes 
i 
2 
% 
Evaluate determinant 
D = \ajj| 
4 


Step I 
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4. DA is the absolute value of the trial determinant D. 


Srrep III 
Compare the computed absolute value of D with the error term E. If itis 
sufficiently small, that is, D < EH, the current value of \ is acceptable. 
If, however, D > E, the trial value of \ must be increased and the process 
repeated. 

A flow diagram for carrying out Step III is shown in Fig. 7.12. The 
indicated notes correspond to the following: 


1. If DA is found to be equal to or less than the allowable error £, 
V is an acceptable eigenvalue. Such solutions are put out as H7gz. 
The variables are next reset, and control returns to step @). If the 
absolute value of the determinant is greater than the allowed error, 
the value of the trial eigenvalue V must be increased and a new 
determinant evaluated. 


2. If V = Val, that is, the first assumed trial eigenvalue is in fact an 
acceptable solution, the only course of action is to increase V by 
s and proceed to the determination of a second eigenvalue. If, 
however, V was not equal to Val, this was not the first cycle and 
the sign of D must be considered to ensure that the root is always 
approached in the positive direction. 

3. At this point, a test is required to determine whether D is positive or 
negative. The sign of D when compared with the sign of Check (the 
last previous computed value of D) determines whether the root has 
been passed or not. If such is found to be the case, V is reduced 
to the last preceding value, s is reduced by 14, and D is replaced by 
the previous D. If the eigenvalue under consideration has not been 
passed, Check is replaced by D, a test is made for the maximum value 
of V, and V is increased by s. Control then returns to @). 

4. If k = 1 when V > 7, no value of \ exists between the assumed 
starting value of V and the terminal value T. If k ¥ 1, all the 
desired eigenvalues have been determined. 


A complete flow diagram for the determination of eigenvalues in the 
manner just described is given in Fig. 7.13 (pages 156-157). 


Prob. 7.8: The flow diagram just developed was based on the assump- 
tion that the problem was formulated according to Eq. (7.68). ~What 
changes in the program would be required to facilitate solution of prob- 
lems of the Eq. (7.67) type? 

Prob. 7.9: Construct a flow diagram for the determination of the 
smallest eigenvalue, and its corresponding eigenvector, using iteration. 


eigenvalue problems 155 


Notes_ 


Out 
No eigen- 
value in 
Step II 


Figure 7.12 
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Evaluate determinant 
D= lay; 


Figure 7.13 Flow diagram for determination of eigenvalues. 
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Figure 7.13 (Continued) 


nonlinear equations 


8.1 Introduction 


By definition nonlinear equations are assumed to be those 
which contain powers or products of the variable(s) and/or 
transcendental functions. It is further presumed that tran- 
scendental encompasses all nonalgebraic situations, such as 
Sin 2) COS 4, eo mee, and J (a). 

Two problem types are of primary concern. The first is 
the evaluation of the “roots” of a given equation. The 
second is the determination of those particular values of the 
variables which simultaneously satisfy a given set of nonlinear 
equations. 

Except for very special cases, no general algebraic methods 
are available for the solution of these types of problems. 
Under certain situations it may not even be possible to estab- 
lish mathematically whether a solution (or solutions) does or 
does not exist. For these and other reasons, most solutions 
of nonlinear equations are obtained by iterative methods. 
As will be demonstrated later, to ensure convergence of these 
methods it is necessary that a reasonable approximation to 
the solution in question be known prior to iterating. 

It should be understood at the outset that roots of non- 
linear equations may be real or complex. While the major 
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part of this chapter is concerned with methods for obtaining ‘“‘real-root 
solutions,” Sec. 8.5 discusses one method for obtaining complex roots of 
nonlinear algebraic equations. 

Solution of problems of the type considered in this chapter normally 
consists of two phases: obtaining a first approximation, and improving 
the accuracy of the solution. 


8.2 Methods for Obtaining First Approximations 
8.2.1 Graphical Methods 


One of the most frequently used scanning methods to establish first 
approximations to the solutions in question is that of sketching the 
curve(s) in the region desired. For example, consider the determination 
of the smallest real positive eigenvalue for a column fixed at its base and 
pin-connected at its uppermost point. The equation for which roots must 
be found is 

F =tanrx—A =0 (8.1) 


Graphically, F can be plotted (or sketched) as a function of \, as shown in 
Fig. 8.1. From this figure, it is evident that a first approximation to a 
positive-value solution other than zero would be } = +4.5. It should 
be noted that while the general shape of F may (or may not) be readily 
apparent for this one particular given equation, such will not in general be 
true. Even for this case, possibly an easier scanning could have been 
realized by introducing a third variable z and separating the given equa- 
tion (8.1) into the two equations 


2 = tan A 2=h (8.2) 


A simultaneous solution of these more elementary equations, the general 
shapes and magnitudes of which are known, corresponds to the actual 
solution of Eq. (8.1). Thecurves areshown in Fig.8.2. Again, \ = +4.5. 


Figure 8.1 
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Figure 8.2 


Approximations to the roots of most nonlinear equations can be 
obtained in this same manner. Consider the cubic equation 


F = x3 — 277+ 24 —1=0 (8.3) 


A direct plot of this equation is shown in Fig. 8.3. To facilitate solution, 
the original Eq. (8.3) can be rewritten in terms of a new variable y, where, 
by definition, 

r=yth (8.4) 


and h is a constant to be chosen at a later stage. Equation (8.3) can 
therefore be written as 


(yar hh)? — 2g +h)? + 2@ hy — F—0 (8.5) 
or 


y? + y2(8h — 2) + y(Bh? — 4h + 2) + (A? — 2h? + 2h —1) =0 (8.6) 


Figure 8.3 


[ | 


O +1.0 
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O =n O)3) 


Figure 8.4 


By selecting h = +24, it is noted that the coefficient of y? can be made 
to equal zero. The resulting equation is 


y? + yg) — ka7 = 0 (8.7) 

which can be readily separated into the two more elementary equations 
Baw 2 fog — 18y) (8.8) 

These two curves are sketched in Fig. 8.4. The solution is y = +14, or 
tea lt ae pete) me oO (8.9) 


Additional information is also evident from Fig. 8.4. Since for plus values 
of y, y® is a continuously increasing function; since for negative values, it 
has an increasing negative value; and since z = 147(7 — 18y) is a linear 
function of y, only one real root of Eq. (8.3) can be found. 

_ As a third illustration of the graphical determination of first approxi- 
mations to nonlinear equations, consider the two simultaneous nonlinear 
equations 


gay) = 3y* — 223 — 1027? +41 =0 


(8.10) 
GG We= We ate oy == 0 
One obvious method of solution is to plot ‘‘contour maps” of each equa- 
tion and superimpose them. By noting that the first equation contains 
the variable y in only one term and that a similar situation for x exists in 
the second equation, Eqs. (8.10) can be written as 
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Figure 8.5 


filzy) =y = = TI ap 


gy) == jw i = al 


(8.11) 


These equations can be sketched as shown in Fig. 8.5. In the region 


shown, two solutions to the original set of equations exist. 


8.2.2 Special Methods for Approximating the Real 
Roots of Nonlinear Algebraic Equations 


The graphical procedure just described can be used to obtain approximate 
values of the roots of almost any nonlinear equation. However, when the 
equation is completely algebraic (i.e., a polynomial in x), there exist 
certain characteristics which can be of considerable aid in finding the 
approximate values of certain roots. 

The algebraic equation in question is 


f(x) = nx” + Gp_iv"—! + Gnoge™? + + + + + are + ao 
In general, it has n roots. These may be real and distinct, real and 
repeating, or imaginary, in which case they would appear in couples of 


complex conjugate numbers. 
Those real roots less than 1 in absolute value can be found by evaluating 


f(x) at intervals of 0.2 or 0.1 in the range —1 < x < +1 and sketching 
For real roots less than —1.0 or greater than +1.0, 


=0 (8.12) 


the resulting curve. 
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F(x)=4x5—-10x2-2x+6 
+10.0 4 ] 


hima in 


See 
{4 | 


—1.0 O +1.0 


Figure 8.6 


it is desirable to rewrite Eq. (8.12) in terms of a new variable z, where 
1 
7 (8.13) 
Z 
The equation is 

Pye Oe Oe” dae 4 Gn = 0 (8.14) 


and the range of examination is —1.0 < z < +1.0. 


Figure 8.7 
f(z)=6z3-—222 -107+4 


10.0 
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For illustration of the general method, consider the polynomial 


f(z) = 4c* — 10x? — 22 4+ 6 = 0 (8.15) 
Solutions are sketched in Figs. 8.6 and 8.7, and the approximate roots are 
= —0.76 xz = +0.80 z= +2.50 (8.16) 


Alternative analytical methods for obtaining approximations to certain 
real roots are given by the following: 


1. The largest root of a polynomial of the Eq. (8.12) type can be approxi- 
mated by the larger of the roots of the following two equations: 
i+ Gra = 0 (Sid) 
or Ant? + Gn1t + Gn_2 = 0 (8.18) 
For the previously considered polynomial, 4z* — 10x” — 2x + 6 = 0, 
the equations corresponding to (8.17) and (8.18) are, respectively, 
4a? — 10% — 2-=.0 


The root of the first of these is x = +2.5. The larger root of the 
second equation is 7 = +2.69. 

2. The smallest root of a polynomial of the Eq. (8.12) type can be 
approximated by the roots of the following two equations: 


(8.19) 


az + a = 0 
(8:20) 
or aoe? + ax + a = 0 
For Eq. (8.15), these are, respectively, 
—24+6=0 
(8.21) 


—10x72? — 2x +6=0 


The roots are x = +3.00, from the first of these equations, and 
xz = —0.98 and +0.58, from the second. An approximation to the 
smallest root is therefore x = —0.98. 


8.2.3. Relationships That Exist between Roots of Algebraic Equations 


As a guide (but not foolproof), the number of possible positive and nega- 
tive real roots of equations of the type defined by Eq. (8.12) can be deter- 
mined from Descartes’s rule of signs: 


1. The number of positive real roots is either equal to the number of 
variations in sign of the coefficients of the equation or less than that 
by a positive even integer. 
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2. The number of negative real roots is either equal to the number of 
repetitions in sign of the coefficients of the equation or less than 
that by a positive even integer. 


Zero coefficients are ignored in thiscount. For illustration, again consider 
the polynomial 


f(x) = 4x3 — 102? — 22 +6=0 (8.22) 


There are two sign changes (+4 to —10 and —2 to +6). Therefore, 
there should be either two or no positive roots of Eq. (8.22). Since there 
is only one sign repetition (—10 to —2), there should be one negative real 
root. These were shown in Figs. 8.6 and 8.7. 

Another general rule concerning the roots of equations of this type can 
be stated. If n is odd (that is, the equation is of odd degree), there will 
be at least one real root and it will have a sign opposite to the sign of 
a)/ad,. Again consider Eq. (8.22). Since it is of odd degree (cubic), 
there must exist at least one real root whose sign is opposite to +4/+6. 

It is possible to develop expressions relating the coefficients of the 
various elements in a polynomial to the roots of that equation. To 
illustrate these, again consider the general equation (8.12), here repeated 
as Eq. (8.23). 


f(z) = Onx? + GO, 10"! + apt"? + -- > +ae+ao=0 (8.23) 
This can be written as 


f(a) = 2" 4 ap ae? + apt”? +--+ or tay=0 (8.24) 


An-i 
where Oni = (8.25) 
If the roots of this equation (that is, v1, 2, . .., %n) are known, Kq. 


(8.23) can equally well be written as 
f(x) = (x — a1)(x — a2)(a — 4s)(- - ‘)(% — tn) = O (8.26) 


Expanding Eq. (8.26) yields the equation 


f(z) =a" — Oo} Oy ees (y ts ) a ( 


Lilie) gr-3 ob ao 8 
1 
1 


(—1)"(awersts °° tn) = 0 forti#x¥ jp AkA--- (8.27) 


Since by definition Eqs. (8.24) and (8.27) represent the same polynomial, 
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the coefficients of each must match. That is, 


n 
Q2-1 = — ») Xi 


n 
Ong = + y CH; 


is (8.28) 
Ons = — ) wits 
Si 
k=1 
ao = (—1)"(a1tor324 tees La) 


where? #j ALF - 


8.2.4 Synthetic Division 


In determining the real roots of polynomials, or for that matter in finding 
the numerical value of any given nonlinear algebraic function at a pre- 
selected value of the independent variable, a process known as synthetic 
division may be useful. It should be here stated also that the process can 
be used to establish the numerical values of the derivatives of the given 
function at the preselected point. The relationship that exists is defined 
by the following: 


Remainder of first synthetic division = f(b) 


iL 
Remainder of second synthetic division = 7 f’(b) 
é : Eops! ts 1 
Remainder of third synthetic division = i f’’(b) (8.29) 


Remainder of fourth synthetic division = + 7 (b) 


where 6 is the preselected point. 

For illustration of the general procedure to be followed, consider the 
cubic equation given in (8.30). (The algebraic expressions for the deriv- 
atives are tabulated also for future reference.) If it is assumed that the 
value of x in question is 0.6, that is, x = +0.6, the numerical values of 
the function and its derivatives at this point are those given in Eqs. (S34) 
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f(z) = 23 + 52? + Or — 2 (8.30) 
f'(x) = 32? + 102 
f(x) = 6x + 10 
f(0.6) = +0.02 
f'(0.6) = +7.08 (8.31) 
f’(0.6) = +13.6 


Consider, now, a long-division evaluation of the function, presupposing 
x = +0.6, or z — 0.6 = 0. 


+ w+ 5.62 + 3.36 


zx—O6\7?+ 52? + Ox — 2 


xz* — 0.62? 
ao Ot OF (8.32) 
+ 5.62? — 3.362 
+ 3.362 — 2.00 
+ 3.362 — 2.02 


+ 0.02 = remainder = (0.6) 


Note that the remainder exactly equals the tabulated value for f(0.6) 
given in (8.31), which was obtained by direct substitution. Dividing 
the quotient of Eq. (8.32), that is, the quadratic 


x* + 5.6” + 3.36 
by x — 0.6 will give as a remainder the value of the first derivative f’ (0.6): 


x 46.2 


ti 0.6|-+- 2" + 5.67 -F 3.36 


xz? — 0.6x 


+ 6.24 + 3.36 (8.33) 
+ 6.20 — 13.72 


+ 7.08 = remainder = f’(0.6) 
Again, dividing the “new” quotient by x — 0.6, 


+f 


x — 0.6lz + 6.2 (8.34) 


xz — 0.6 
+ 6.8 = remainder 
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According to Eq. (8.29) the remainder of this third synthetic division 
equals 1¢f’’(b). Therefore, 
Yef'(0.6) = +6.8 
f’ (0.6) = +13.6 (8.35) 
A more convenient form for synthetic divisiom is shown in (8.36). In 
this formulation, the x” terms are omitted and only the coefficients listed. 
(It should be noted that zero coefficients must be included when a given 


powered term is absent from the polynomial.) The original polynomial 
x? + 5x? — 2 is written as 


1 +5 +0 —2 
In “short form,” the first synthetic division is 


i) ee 0 =2 
ti = 0.6 0.66 os 36 0 P20 (8.36) 


1 +5.6 +3.36 |+.0.02 = f(x) = f(0.6) 


This same procedure can be continued to establish the values of each of 
the derivatives. The complete set is given as (8.37): 


io +0 —2 
t1 = 0.6 +0.6 +3.36 +2.02 

1 +5.6 +3.36 |+0.02 = f(z) (8.37) 
t1 = 0.6 +0.6 +3.72 

1 +6.2 |+7.08 = f’(21) 
t1 = 0.6 +0.6 


1 
1 |+6.8.= 21 f’’ (a1) or t" (a1) = +13.6 


8.3 Methods for Improving the Accuracy of a Trial Solution 


First approximations to the real roots of a given nonlinear equation having 
been obtained, it is desirable to consider certain of the more common meth- 
ods for improving these trial values. 


8.3.1 False-position Method 


Assume, for example, that the function values f(x) for two relatively close 
trial values of x are known and that one of the function values is positive 
while the other is negative. Obviously, at least one root will be contained 
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F(x) 


True 
solution 


Figure 8.8 


within this range of x. Figure 8.8 illustrates this situation. The two 
trial points correspond to 21, f(x1) and xo, f(x»). 

An improved approximation to x corresponding to f(x) = 0 can be real- 
ized by linearly interpolating between these two points, as shown by the 
dotted lines in Fig. 8.8. The governing equation is 


C= Ca fa Core Cy 
f(x) — F(a) is f(t2) — f(x) 


where x and f(x) correspond to any point on the line. For that par- 
ticular point where x = x3, and for which f(x;) = 0, 


atif(x2) — tof(x1) 
f(t2) — f(x) 


For the case illustrated in Fig. 8.8, f(xz3) also has a negative value. There- 
fore, the next linear interpolation would be between 22 and 23. 

As a numerical example, consider the cubic equation (8.3), which is 
repeated here as Eq. (8.40): 


(8.38) 


L3 = (8.39) 


fia) = 23 — 227 + 22 —1=0 (8.40) 


From Fig. 8.3, it is noted that « = +1.0; however, to illustrate this 
method of linear interpolation (or, as it is often referred to, the method 
of false position), it will be assumed that the solution is contained within 
the interval +0.9 < x < +1.1. By synthetic division or direct substi- 
tution, the following values are obtained: 


41.1 f(a2) = f(1.1) 


—0.091 
+O: 11 


l| 


(8.41) 


I 
ll 


v2 
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From Eg. (8.39) the value of x3 for the “next trial” is 


_ (+0.9)(+0.11D) = (41.1(— 0.091) _ +0.2000 _ 9 999 (8.42) 


X3 


(+0.111) — (—0.091) ~ +0.202 
Substitution of this value in Eq. (8.40) yields 
f(zs) = f(0.99) = —0.0099 ; (8.43) 


Interpolation is therefore between x2 and 23, and x4 is 


mea Xsf(e2) — Bef(ws) _ (+0.99)(+0.111) — (+1.1)(—0.0099) 


f(t2) — f(xs) £0,111) — (— 0.0009) 
+0.9992 (8.44) 


This process would be continued until the desired accuracy has been 
achieved. 


8.3.2 Newton’s Method of Tangents 


A second iterative method for improving the accuracy of a trial solution, 
known as Newton’s method of tangents or just Newton’s method, can also be 
used. In most cases it will result in faster convergence than will the false- 
position method. In addition, it requires as a starting situation a con- 
sideration of only one point. 

Consider Fig. 8.9. It is assumed that x1 is reasonably near the true 
solution and that f(x) is the function value at that point. A second 
approximation can be realized by considering the tangent to the curve 
f(x) at point x; and reducing the x; value accordingly. This is shown by 
the dotted line. The equation for the next approximation is 


_ Fn) 

F' (tn) 
where f’(,) is the first derivative of the function value evaluated at 
x =2n. (The student is urged to verify the correctness of this equation 


Lnt1 = Ln (8.45) 


Figure 8.9 


True solution 
F(x) 
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for positive or negative values of the function value as well as positive 
and negative first derivatives.) 
Again, for the problem just considered, 
f(x) = 23 — 22? + Or —1 =0 (8.46) 


it will be assumed that x1 = +1.1 is a reasonable first approximation. 
The first derivative with respect to 2 is 


f(z) = 32? —42 + 2 (8.47) 


The values to be used in Eq. (8.45) are therefore f(1.1) = +0.111 and 
pais) 9st! 1 193: 
Oli 


(ete 1 0, Se. 
- Tag ~ Ti — 0.0902 = +1.0098 (8.48) 


Several cycles of approximation are given in Table 8.1. 


Table 8.1 
Approx (n): 1 2 3 
Ln +1.1 +1.0098 +1.0001 
ite) +0.111 +0.0099 
f' (zu) +1.23 +1.0198 
f(&n) 
+0.0902 +0.0097 
Pee), 


8.3.3 Newton’s Second-order Method 


A third method for achieving even faster convergence is that known as 
Newton’s second-order method. If it is assumed that the function f(z) 
varies continuously over the region in question, a Taylor series can be 
written with respect to a given point zn. The function value at a new 
point h away from 2, (that is, at tn41) is defined by 


flanss) = Hen) + fen + ne (8.49) 


where Lac Leet (8.50) 


By using only those terms of the series which are shown in Eq. (8.49), 
and assuming that the new function value under these conditions is the 
correct solution, that is, f(zn41) = 0, the following is obtained: 


flan) +h] fen +n] =o (8.51) 
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For the h contained within the bracketed term, it will be assumed that the 
first-order Newton method, repeated here as Eq. (8.52), will hold. 


f(&n) 
Ln41 = Ln — We) (8.52) 
That is, 

f(tn) 
h = %eu1 — fn = = fGen) (8.53) 

Substitution of Eq. (8.53) into Eq. (8.51) yields 

PCO RICD - 

F(a) +h E Cale = GD \e 0 (8.54) 
or —_ ts) a3 as) (8.55) 


h fen) 2 f’ (an) 


For the cubic x3 — 2x? + 2x — 1 = 0, the solution using this method in 
the vicinity of x = +1.1 is given in Table 8.2. 


Table 8.2 
Approx (n): 1 2 3 
In +1.1 +1.001 —1.000 
f (an) +0.111 +0.0010 
f'n) +-1.23 +1.0020 
f"' (an) +2.60 +3.0060 
1 
; =105115 —1003.0 
h —0.099 —0.0010 


As was mentioned earlier in this section, it is frequently advantageous 
to define algebraically the derivatives of the equations in question and 
from these determine the function value and the value of the derivative 
at an assumed point rather than to use synthetic division. A symbolic 
notation for cumulative multiplication also can be used to computational 
advantage. Consider the following equation: 

f(x) = «® — 1825 + 81x4 — 100z? + 40x? — 202 + 4 =0 (8.56) 
The first and second derivatives are given as (8.57) and (8.58): 
f'(a) = 62° — 90x4 + 32423 — 300x2 + 80x — 20 (8.57) 


f(x) = 30x* — 360x3 + 9722? — 600x + 80 (8.58) 
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Introducing the symbol ], which means that everything to the left of the 
symbol is to be multiplied by the quantity immediately following it, the 
three equations (8.56) to (8.58) can be written as 


f(x) = x — 18] x + 81] x — 100] x + 40] z — 20] 4 +4 (8.59) 
f(x) = +6] x — 90] x + 324] x — 300] 2 + 80] x — 20 = (8.60) 
f(x) = +80] « — 360] x + 972] x — 600] x + 80 (8.61) 


It should be noted that such a formulation is efficient for both calculators 
and computer and can be carried out on most desk calculators without 
copying intermediate entries. Solution will be sought in the region 
+4.0 <x < +5.0. The function values at these points are 


f(+4.0) = +564.0 f(+5.0) = —1,596.0 


Solutions using each of the three methods are given in Tables 8.3 to 8.5. 


Table 8.3 Solution by False Position 


Approx (n): 1 2 3 4 5 6 
iy +4.0 +5.0 +4.26 +4.38 +4.43 +4.435 
flats) +564.0 —1,596.0 +313.2 +136.6 +50.1 


Table 8.4 Solution by First-order Newton’s Method 


Approx (n): 1 2 2 A 5 
se 440 +4. 855 44.551 WY wer, 
flan) +564.0 1,048.32 193.41 13.441 40,011 
Ff! (an) — 660.0 3,444.85 = 2,993.48  —1, 922.38 
2 = =0. 855 +0.304 +0.087 +0.007 
"(tn 


Table 8.5 Solution by Second-order Newton's Method 


Approx (n): 1 2 3 4 
fac nee ees ee ee 
dip +4.0 +4.152 +4.390 +4.459 
f(a) +564.0 +437 .7 +120.0 —3.764 
yf (Be) —3, 540.0 —1,012.0 —1,642.8 
ii Ge) —2,128.0 —1,786.8 —2,416.7 
h +0.152 +0.238 +0.069 


ee _O— 
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8.3.4 Direct Iteration 


Direct iteration can also be used to improve the accuracy of a trial solu- 
tion for the real roots of a nonlinear algebraic equation. In essence, this 
procedure requires that 

f(z) =0 (8.62) 
be replaced by 


Iteration is accomplished by substituting into the ¢(z) function the values 
of a trial x; that is, the (k + 1)st trial value of x is obtained by substitut- 
ing the kth trial value of x in ¢(z). 


rit = (x2) (8.64) 


To facilitate convergence, it is necessary that the absolute value of the 
first derivative with respect to x of the function ¢(x) be less than 1.0. 
Since in general there will be several possible ¢(z)’s that will satisfy this 
condition, the function whose derivative has an absolute value which is 
least in the range in question will be the most suitable. 

Consider again, for illustration, the cubic equation (8.46), here repeated 
as Eq. (8.65): 

x? — 227 ++ 24 —1=0 (8.65) 


Five elementary ¢(x) functions can be found for this equation. These 
are given as Eqs. (8.66) through (8.70). 


—r3 2 
Pies Semmes Deny ea ae =e oe (8.66) 
3 — 
20? = 23 + 2a — 1 or g2(x) = qcee ES (8.67) 
x2(2)(¢ — 1) = 2 — 1 or $3(2) = sce (8.68) 
2(a — 1) : 
a? = 222 — 24+ 1 or oi(z) = VW 2x? — 22 +1 (8.69 ) 
1 
2 oy 2) = I ee 
(a x + 2) or $35(x) ReGen (8.70) 


Expressions for the derivatives of these functions with respect to x are 


given as Eqs. (8.71) through (8.75). Values have been calculated for 
Fp call bel 


1 
Geog (8 + 4a) gi(11) = 40.885 8.71) 


_ 9$() 1 3a. +2 
de 4V/x8 4+ On — 1 


$,(1.1) = +0.885 (8.72) 
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0¢$3(2) * 1 2x? + 32? + 1 


$,(2) = a agai: $4(1.1) = +1.600 (8.73) 
, Oda(x) 1 Ay — 2 
4 = = (1.1) = 642 : 
(x) aE ion oe @,(1.1) = +0.642 (8.74) 
¢.(a) = Ose 1) =, (1.1) = —0.186 (8.75) 


ox (x? — 2x + 2)? 


From the above, $5(zx) is the most desirable iterating function. Several 
cycles using it result in the following: 


stp lee 0.9957-1- 1-000, +1000... 2: (8.76) 


The first three cycles using $3(x) are shown in (8.77). It should be 
remembered that this latter function does not satisfy the condition that 
¢' (x) < 1, and therefore it is not a suitable iterating function. 


ele lge 1. G00," 3.400, 6m & (8.77) 


8.4 Real Roots of Transcendental Equations 


Transcendental equations, like algebraic equations, may exist as single or 
simultaneous equations. They may also have real or imaginary roots. 

In general, all the methods described in the preceding parts of this 
chapter are directly applicable to the solution of transcendental equa- 
tions. In addition, the transcendental elements of a given equation(s) 
can be represented, to the accuracy desired, in series form and thereby 
reduce the system to nonlinear algebraic equation(s). Table 8.6 sum- 
marizes a few of the more frequently referred to series. 


Table 8.6 
Function Series 
ae ag x4 
oe rag 3 irsame rity in sag 
: 3 x a 
sin x xu 31 AP Bl 71 
fig coe’ 
cos x i aI AF a6! 
a? 2a> iN pfs 
tan x Dee ge gore sae 


176 modern methods of engineering computation 


For illustration of the method of solution of transcendental equations 
by expansion in series form, consider the determination of the smallest 
positive root of Eq. (8.78). 


Ge ome —tanz =0 (8.78) 


By replacing tan x by its series (from Table 8.6), Eq. (8.78) can be written 
as 


1—2z 4p ge 1727 
ee Sa =e bee. () 8.79 
f(z) : (@+o+ e+ ot ) ( ) 


For an approximate value of the root, consider only the first term of the 
series, that is, 


fe (8.80) 
x 
or xv+a«—-1l=0 (8.81) 
which has roots 
x = +0.56 and —1.62 (8.82) 


A more exact answer can be obtained by considering the first three 
terms of the tangent series. 


1l—2z Geo Dap 
f(x) 1? (424%) ~ (8.83) 
or 20° > 5a*-p Toa =- on = 15 (8.84) 


This can be written in the iterative form of Eq. (8.63) by factoring from 
the left-hand side of the equation an z, as shown in Eq. (8.85): 
15 
Te) 55 beh-F 1h E15 te 
Since the value of the derivative with respect to x of ¢(x) at x = +0.56 
is much less than 1.0, iteration should converge. Several cycles are as 
follows: 


+-0:56, +-0.615, --0.587, +0601, 3. (8.86) 


Direct application of Newton’s method to the equation in its original 
form can also be carried out to obtain a solution to this problem. The 
appropriate expressions are 


; 1 - 
fe) =~ - tanz = 0 


1 (8.87) 
7, (a) = 2 — sec? x 
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By noting that sec? x = 1 + tan? 2, the iterative form of the equation is 


ae — 2071 + tan 22) 
C= fo — 8.88 
oe : ee (he tan) ( ) 
Table 8.7 summarizes several cycles of iteration. Graphically, the equa- 
tions are plotted in Fig. 8.10. Only the region —1.5 < x < +1.5 is 
shown. 


Table 8.7 

k 0 1 2 

t +0.56 +0.595 +0.596 
= —0.035 —0.001 


8.5 Complex Roots of Nonlinear Algebraic Equations 


The general polynomial equation under consideration is that listed orig- 
inally as Eq. (8.12) and here repeated as Eq. (8.89). 


Oye On ot” 4 ey — 0 (8.89) 


It is assumed that both real and complex roots may exist. 


Figure 8.10 


Bima e oe 
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By presupposing that the real roots have been established (for example, 
t1, @, etc.), the original equation can be reduced in order by dividing 


synthetically from Eq. (8.89) the values z — 21,2 — %2,.... Thepol- 
ynomial remaining after these expressions have been removed is 
g(x) = dna” + Dnair™! + --- + biz + bo = 0 (8.90) 
Complex roots must exist in conjugate pairs of the form 
Gar 0s (8.91) 


Therefore, the product 
[eae tere) a Cee) 

must be a factor of the equation in question. Expanding this relationship, 
2 era (Cie & Jen (8.92) 


Equation (8.92) is a quadratic in x having real coefficients. 
Should it be possible to find a quadratic equation of the general form 


2?+us+v=0 (8.93) 


which is an exact divisor of the original equation (8.89) [or (8.90)], the 
roots of that quadratic will represent two of the complex roots in question. 
The problem, then, is to develop a method for determining suitable values 
of u and v. 

If it is assumed that certain trial values for uw and v are known, the 
given polynomial f(x) of Eq. (8.89) can be written as in Eq. (8.94). 


f(x) = (@? + ux + v) (ene? + Cae? 4+ + + + + em + co) + cre + Co 
(8.94) 


For the correct values of wu and 2, c and cy will equal zero. 

The coefficients Cn, Cn1, . . . , C3, C2 in the reduced equation can be 
directly related to the original coefficients an, Gn1, . . . , @1, @o and the 
trial values u and v. Expanding Eq. (8.94), 


Cat” = (Cn = ea)? (Cae Cait Cae) 
+ (€n-10 + Cro + Cn—s)a™-3 + + > > + (ev + cow + c1)r 
+ (cv +) =0 (8.95) 


A comparison of the coefficients is given in Table 8.8. 
To illustrate the method of solution using the relationships of Table 8.8, 
consider the solution of the cubic equation 


rei 0 (8.96) 


For this case a3 = +1, a2 = 0,4; = +1, anda = +1. The object is to 
select u and v so that c, and cy approach zero. While there can be developed 
iterative methods for the solution of the two-independent-variable type 
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Table 8.8 
Qn = Cn Cn = An 
Qn-1 = Ch + Ch_4 Cn—1 = Gn-1 — Cpl 
Gn—2 = Cn) + Chil + Cn-2 | Cn-2 = An-2 — Cris — Cyd 


Qn-3 = Cn—iv = Cn—2U ++ Ca=3 | Cn—3 = Gn—3 — Cnu—9 — Croid 
a, = c3v + eu + eC Cy = A, — Col — C30 


Ay = CW + Co Co = Ao — Cov 


of problem, a trial-and-error approach will be demonstrated for this 
example. To facilitate investigation, the form of Tables 8.9 and 8.10 
will be used. 


Table 8.9 
Computed values of cp 
U 
v 
a; = +1 cs = a3 
a= 0 Co = Ae — C3U 
a, = +1 Cy = Ay — Cou — C3V 
ao = 2 Co = Ayo — Cov 


The results of trials 1 through 9 are plotted in Fig. 8.11, where the 
ordinate is the function value of either c; or ¢o, and the abscissa is wu. 
Based on Fig. 8.11, trial 10 was carried out. By assuming that c; = —0.01 


and cy = —0.05 are sufficiently close to zero, the complex roots of Eq. 
(8.96) will correspond to the roots of Eq. (8.97). 

2? — 0.74 +1.5=0 (8.97) 
That is, 


2 = +0.35 + V(0.35)? — 1.5 = +0.35 + 1.17 (8.98) 


These same values could have been obtained more directly by noting the 
relationship between u and v and ¢ and & [see Eqs. (8.92) and (8.93)]. 
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Table 8.10 
Computed values of cy, 
if 8 9 10 
U —0.5 —0.8 —1.0 —0.7 
v +2.0 +2.0 +2.0 +1.5 
C3 +1.00 +1.00 +1.00 +1.00 
C2 +0.50 +0.80 +1.00 +0.70 
Ci —0.50 —0.36 0 —0.01 
Co 0 —0.60 —1.00 —0.05 
Since 


—2¢ =u = —0.70 ¢ = +0.35 
and C+ 2=y= 41.5 or &= 4+1.17 
Therefore, from (8.91), 

= (+ 7 = +0.35 + 1.1% 


8.6 Real Roots of Simultaneous 
Nonlinear Algebraic Equations 


(8.99) 
(8.100) 


(8.101) 


The methods discussed earlier in this chapter for finding the roots of a 
single nonlinear algebraic equation in one independent variable are not 
directly adaptable to those cases where more than one equation is speci- 


Figure 8.11 
| | | +1.0 
==—==V=+ 1:0 | 
se | 
ae ero) a Function 
value 


solution ¢,=Co) 
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fied. The methods can, however, be modified to take care of these cases, 
as will be demonstrated in this section. 

For illustration, the two-independent-variable set of two nonlinear 
equations defined by (8.102) will be considered. 


Larger-order systems would be examined in a similar manner. Two 
methods of solution will be considered: (1) Newton’s method and (2) 
direct iteration. 


8.6.1 Newton’s Method 


By assuming that a sufficiently close approximate solution x = a, and 
y = b, is known, Eqs. (8.102) can be expanded into Taylor series about 
these points. By using h and k to represent the changes in x and y, 
respectively, which need to be made in successive cycles of approximation, 
the expansion of Eqs. (8.102) can be written to a first-order approximation 
as 


f(ai + h, by + k) = f(ai,b1) + hfz(ai,b1) + kf,(ai,b1) = 0 


(8.103) 
g(ai + h, b1 + k) = g(ai,b1) + hgz(ai,b1) + kgy(a1,b1) = 0 
where 
Of(ai,b Of (a,b 
(ay RE Cn es 
Ox oy 
Ges (8.104) 
0g(a 1b og a, 
Gz(@1,b1) = “esGuts Jy(d1,b1) = cits 
and y= a4 + h y=bt+k (8.105) 
Equations (8.103) can be solved simultaneously to define h and k. 
fu — F9y 
= —————_—— (8.106) 
Sey — Gay 
a SG: — Ge (8.107) 
Foy — Gaby 


where the function values and their derivatives are evaluated at x = a; 
and y = 6. 
For illustration of the procedure, consider the following two simultane- 
ous nonlinear algebraic equations: 
{G@y) = 3y? — 273 — 102? +1 = 0 


GG) = yo 8a? oy 10 = 0 


(8.108) 


182 modern methods of engineering computation 


The various required derivatives are 
fe = —6x? — 202 Jz = —6z 
jn = +99" gy = +3y2+ 5 


(8.109) 


Table 8.11 gives the solution, assuming as approximations x; = +0.6 and 
Ui = +1.0. 


Table 8.11 
Cycles: 1 2 3 
iG +0.6 +0.606 +0.606 
y a0 +1.013 +1.013 
f —0.032 +0.001 
ifs —14.160 — 14.323 
ti +9.000 +9.235 
g —0.080 +0.003 
9 —3.600 —3.636 
Iu +8.000 +8.078 
h +0.006 —0.00023 
k +0.013 —0.00047 


8.6.2 Direct Iteration 


For direct iteration instead of Newton’s method, it will be assumed that 
the two equations can be written in the form 


f(x,y) =x — (x,y) = 0 


(8.110) 
g(x,y) = y — v(a,y) = 0 


By presupposing sufficiently close approximate solutions x = a, and 
y = by, iteration can proceed from the equalities 


rit = (ny) 
(8.111) 
ey = W(na,ny) 


It should be noted that k is used here to designate the cycle of iteration, 
as was done in Kq. (8.64); it is not used to designate the change in y, as 
was the case in Eqs. (8.105). 
Many functions $(2,y) and ¥(2,y) can be found for a given set of non- 
linear equations. Guest} has shown that for convergence, it is desirable 
t J. Guest, On the Choice of Suitable Functions for Solving Non-linear Equations by 


Iterative Procedures, Aeron. Res. Lab., Struct. Mater. Note 262, Australian Defense 
Scientific Service, Melbourne, November, 1959. 
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to select functions which satisfy the following conditions: 
d¢| | d¢ dy oy 
Ox oy ax | oy 


dp 0p APdp dd dp 
Ox Oy Oy Ox Ox oy 


e 
(8.112) 
and 


In addition, since the first of Eqs. (8.110) will essentially be used to define 
z while the second equation will determine y, it is desirable to have 
|9¢/da| and |dy/dy| as small as possible. 
Consider the two simultaneous nonlinear algebraic equations (8.108) 
here repeated as Eqs. (8.113). 
ea no, 2 ~ 
(eu) = 3y 2x 1027 = 1 = 0 (8.113) 
Go — 32" + oy? — 5 0 


From these, by considering Eqs. (8.110), the following elementary func- 
tions can be written: 


>. ee = 203 +1 7 Na + 102? — 1 
gi = 10 vi = 3 


A — 1077+ 1 


$2 = ; vo = V30? — By +5 
pe sm 5 ae wo (8.114) 
se i ee ne Jt Seis 


Since ¢; is a function only of y and y varies only with z, one may be 
tempted to assume without checking that iteration using these functions 
will converge. Such is not the case. 


dbs ads 


Ox Oy 

oy oy 

ee Tbe — =r () (8.115) 
Ox i dy 


db dp AP dp AG Op 
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Table 8.12 summarizes the values of the various derivatives at x = +0.6 
and y = +1.0. 


Table 8.12 
a2, e8 ad a Rt 
ox oy Ox oy 
ot i). 8 +0.75 
$2 —17.5 +13.1 
o3 —1.10 +1.34 
pa —0.05 +0.67 
5 0 +2.31 
V1 +1.56 0 
Yo +1.14 —1.58 
vs +0.72 —0.60 
Ws +0.60 —0.34 
V5 +1.04 —2.92 
p4, W4 +0.01 


Op0~ ddbdp Ad dy 


Functions ¢4 and yw, are selected as the iterating functions. That is, 


ai /3y? +1 
a 2x + 10 


(8.116) 
= ge et 
UY HS 
Several cycles of iteration are given in Table 8.13. 
Table 8.13 
Cycles: 0 1 2 3 4 
———————————————— eee 
ay Sp OnG +0.598 +0.607 +0.603 +0.607 
y sell @ +1.0138 +1.008 +1.015 +1.010 


LS 


Prob. 8.1: Using the false-position, Newton’s first-order, and New- 
ton’s second-order methods, determine the smallest positive roots of the 
following equations. Carry the answers to three decimal places. 
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(a) #2 +127 +3=0 
(b) e?°4+ 82 +2=0 
(c) a4 + 23 + 622 —-2+3=0 
(d) 22+ 7? —4¢ = 0 
(e) 2+ 32 —3 = 0 
(f) 225 + x4 + 11x? — 8224+ 7r —3 = 0 
Prob. 8.2: Determine all the roots of the following equations and 
check the results by using the relationships given in Eqs. (8.28). 
(a) #2 + 32 —3 = 0 
(b) x* — 102? 4+ 352? — 502 + 24 = 0 
(c) x4 — 4x3 — 7x? + 227 + 24 = 0 
Prob. 8.3: Determine the smallest roots (in absolute value) of the 
following transcendental equations: 
(a) 1—x+2snz = 0 
(b) 642 —e7 = 0 
(c) 3tan?@+ 2sin@—6@=0 
Prob. 8.4: Determine the roots of the following sets of nonlinear 
algebraic equations in the range 0 < « < +1.0 and0 < y < +1.0: 
(a) x? — 2ry + y? — 22 —2y+1=0 
y? = x3 
(b) 4yx? — 2a +y =0 
Yen ye 10 
(c) W(1+ 2) = 2(1 — x) 
6x? — 2x7 — 102 + 4 = 0 
Prob. 8.5: Find the conjugate complex roots of the following cubic 
equations: 
(a) #2 +47+2=0 
(b) 2? — 2x +12 =0 
(c) x? — 622 + lla —6=0 
Prob. 8.6: The determination of the buckling load of a column fixed 
at its base and pin-connected at its uppermost point requires the solution 
of the equation 


N= HHA) PN 


Determine the lowest three positive roots of this equation. 

Prob. 8.7: Using both Newton’s method and the method of direct 
iteration, determine to three decimal places the square roots of the 
following numbers: 

(a) 2.000 (b) 23.676 (c) 61.384 (d) 147.361 

Prob. 8.8: For the numbers given in Prob. 8.7, determine the cube 
roots by using Newton’s second-order method. Carry the solutions to 
‘hree decimal places. 
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Prob. 8.9: Given the equation 
ie) = V73—-2=0 


and assuming, as a starting value, that the root in question is in the 
vicinity of c = +3.1, carry out the first four iterations using Newton’s 
first-order method. Discuss the convergence of the solution and indi- 
cate how this problem is different from those illustrated in the text. 


8.7 Computer Solutions 


Since, computationally, all the methods for determining the roots of non- 
linear algebraic equations discussed in this chapter are of the same general 
type, only Newton’s first-order method will be illustrated. The others 
would be handled in a similar manner. 

It will be recalled from Sec. 8.3 that Newton’s first-order method for 


Figure 8.12 
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determining the roots of a given nonlinear algebraic equation requires the 
repeated application of Eq. (8.117). 


_ fen) 

F (&n) 
Moreover, an initial value for xn, sufficiently close to the root in question, 
must be given. 

For simplicity and generality, it will be assumed in the flow diagram 
that expressions for both f(x) and f’(x) are available. The approximate 
solution 2 will also be presumed. 

Figure 8.12 is a flow diagram for the determination of real roots of non- 
linear algebraic equations using Newton’s first-order method. The notes 
in the figure correspond to the following: 


(8.117) 


Tn4+1 = Xn 


1. Computation begins with the introduction of the approximate value 
of the root in question, zo. An allowable error in x, which governs 
the accuracy of the desired root, must also be specified. 

2. A, the last calculated value of the root, is first set equal to Zo. 
f(x) and f’(x) at x = A are then evaluated. In the flow diagram, 
FA has been used as the designation for f(A). F1A stands for f’(A). 
It should be noted that if f(z) = 0, the second term of Eq. (8.117) 
will go to infinity. Therefore, in the flow diagram, if F1A = 0, 
computation stops and a new value of x» must be selected. 

3. B is the calculated “next approximation” to the root in question. 

4. If the absolute value of the difference between B and A is found to 
be less than Error, an acceptable root has been obtained. If, how- 
ever, the difference is too large, A is replaced by B and the process 
is repeated. 


Prob. 8.10: Construct a flow diagram for the determination of the 
root of a nonlinear algebraic equation, using the method of false position. 

Prob. 8.11: Develop a flow diagram for the preparation of a table of 
fifth roots, for numbers ranging from 0 to 50.0 in increments of 1.0, using 
Newton’s first-order method. The table should be accurate to three 
decimal places. 

Prob. 8.12: Using the method of solution described in See. 8.6.1, 
develop a flow diagram for the determination of the real roots of a set of 
simultaneous nonlinear algebraic equations. 


The first part of this text, Chaps. 1 to 8, is concerned pri- 
marily with the manipulation of numbers and groups of num- 
bers and/or symbols, the emphasis being directed toward the 
solution of a given equation or set of equations. It is evident 
from a consideration of the Table of Contents that a major 
part of the second portion, Chaps. 10 and 11, is concerned 
with the development of methods for the numerical solution 
of differential equations. Many important numerical prob- 
lems in engineering and applied science do not conveniently 
fit into these categories, for example, interpolation of tabu- 
lated data, curve fitting, and numerical integration. Chapter 
9 is included to cover a few of these “‘other topics.” 


9.1 Difference Tables 


To facilitate presentation and to provide a more compact 
form for future reference, scientific and mathematical data 
are frequently arranged in tabular form. For example, 
tables-of Bessel functions are available which list, for equal 
increments of the argument, the appropriate function values. 
Sines, cosines, square roots, interest tables, and location of 
the position of the moon with respect to the sun and the earth 
are all data that are arranged in this form. 
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In dealing with tabular quantities, three types of situations are nor- 
mally encountered. First, given the necessary mathematical relation- 
ship(s), it may be desired to develop a function table or to extend the 
range of one that has already been defined. Secondly, interpolation 
between function values is often required. The third case, referred to as 
inverse interpolation, has to do with the determination of that particular 
x which corresponds to a given f(x), when the given f(x) does not corre- 
spond identically to one of those tabulated. 

These and other topics associated with the use of tabulated quantities 
can be more conveniently discussed after consideration of the develop- 
ment of a difference table. As is shown in a later chapter, such tables are 
also helpful in the solution of certain types of differential equations. 

The specific equation that is used for illustration is the polynomial 


i lS wa i (9.1) 


It is assumed that the range of interest is 3 < « < 8 and that the incre- 
ment, or interya],is 1. Without consideration of how they are obtained, 
the values of the function at the six points in question are as given in 
Table 9.1. By choosing as a positive system one that assumes that f(z) 


Table 9.1 
x +3 +4 +5 =O +7 +8 
T(z) +32 +54 +82 +116 +156 +202 


increases with z, it is noted that the increment in the function values 
between x = 3 andz = 4is 


Af = (4) — 3) = (4754) — (432) = +22 (9.2) 


That is, the difference between the two quantities is +22. Similarly, the 
increment in the function values at « = 4 and x = 5is 


Afias) = f(5) — f(4) = (+82) — (+54) = +28 (9.3) 

For the total range in question 
Afis-4) 
Af 4-5) 

Af (5-6) | (9.4) 


Af(s-1) 
Afi-s) = 
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It should be noted that these first differences are not constants. It is 
therefore possible to consider the differences (or increments) that exist 
between these computed values. That is, second differences can be 
obtained. 


A’f(s.4-5) = (+28) — (+22) = +6 (9.5) 


where|A2f is read as the second difference in the function value f(z). 


Similarly, 


A?*fi45-2) = +6 


A*fis.6-7) = +6 (9.6) 
A?f (6-7-8) = +6 


Extending the concept, 


A? f(3-4-5-6) = 0 
A®fc4.5-6-7) = 0 
A®f(5-6-7-8) = 0 


(9.7) 


While the development presented above is quite straightforward, a 
more convenient form for presentation of the resulting information is 
shown in Table 9.2. It is to be noted that at each stage the difference 
values are located in the difference table between the two primary values 
in question. That A*f = 0 in Table 9.2 is not unexpected. Consider 
the same situation, but this time in terms of the general locations x, x + 1, 
z+ 2,ete. (See Table 9.3.) 

By noting that A?f = 0, Tables 9.2 and 9.3 can be extended indefinitely 
by simple addition, starting from the right-hand side and working to the 
left, and from the top downward. 


Table 9.2 
x f(x) Af A*f Af 
3 +32 
+22 
4 +54 , +6 
+28 0 
5 +82 +6 
me oF 0 
“40 0 
i +156 +6 
+46 


8 +202 
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Table 9.3 
eee ee 
f(x) Af Af Ass 
eee eee ee ee nt aE oe) e 
£ Si ee ee ae 
+62 + 4 
saat 327 -+- 72+ 6 +6 
+6x + 10 0 
Gist? 322 + 132 + 16 +6 
+62 + 16 0 
z+3 3x? + 19% + 32 +6 
+6x + 22 
z+a4 3x? + 252 + 54 
ee ee ee 8 
Table 9.4 
eee ee es 
z F(z) Af A¢f A3f 
3 +32 
+22 
4 +54 +6 
+28 0 
5 +82 +6 
+34 0 
6 +116 +6 
+40 0 
7 +156 +6 
+465 pe oe (0) 
8 +202 so (+6) 
errs ee tee c(h 52) (0) 
(9) (+254) (+6) 
(+58) (0) 
(10) (+312) (+6) 
(+64) (0) 
(11) (+376) (+6) 
(+70) (0) 
(12) (+446) (+6) 
(+76) (0) 
(13) (+522) (+6) 
(+82) 
(14) (+604) 


Certain checks on the numerical work are readily apparent. [or 
example, the first tabulated value of the first difference (that is, Af(s-4)) 
in Table 9.4 is +22. The total change in Af over the range shown is 
=(A2f), or (10)(+6) = +60. Therefore the ending value in the Af col- 
umn should be +22 + 60 = +82. Similarly, f(3) = +32, 2(Af) = +572. 
Therefore, f(14) = +32 + 572 = +604. 
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Seldom are engineering data encountered in the convenient integral- 
number form of the preceding example. More often, decimal values are 
given and a certain specified accuracy is desired. Consider the same 
polynomial, evaluated in the range 3.00 < x < 4.50. The increments 
are to be 0.25, and two decimal places are to be carried in the tabulation. 
(See Table 9.5.) As is customary in a difference table of this type, deci- 


mal points have been omitted. 
pS ns SE A 


Table 9.5 
x f(z) Af A°f A‘f DECIIAL 
3.00 +32.00 FoInTs 
+493 rT 
3.25 +36 .93 +39 OMITTED 
+532 = 
3.50 442.25 +36 
+568 “3 
3.75 +4793 +39 
+607 = 
4.00 +54.00 +36 
+643 3 
4.25 +60.43 +39 aan 
+682 ng ew we 
4.50 +67 .25 iE 


It should be noted that the third difference in Table 9.5 is not zero, as 
was the case in Table 9.4. Moreover, higher-order differences such as 
A‘f and A’f are also not of value zero. Since A’f is constant in magnitude 
and alternating in sign, the difference table could be developed indefi- 
nitely, and nonzero values would continue to be obtained. This situation, 
however, does not rule out the possibility of ‘‘additive extension” of the 
table. In view of the repetitive, alternating nature of the values in the 
A*f column, the table can be extended, as shown in Table 9.6. 

Tor the same polynomial [Eq. (9.1)], Table 9.7 is a difference table in 
the range 3.00 < x < 3.05, where the uniform increment in x is 0.01. 

By comparing Tables 9.4, 9.6, and 9.7, it is evident that the interval 
of the independent variable is of importance in determining the number 
of differences that must be considered to facilitate the extension of a 
function-value table. In general, higher-order differences must be deter- 
mined until either a zero is obtained or the magnitudes of the increments 
are essentially equal and the signs are alternating.) When such situations 


cannot be realized, the interval is too large and a smaller one must be 
‘lated; na oa 
—— 
TE STEADY STATE Dofs NST COE 
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Table 9.6 
SSS ee eee 
x f(z) Af IN NGF, 
SS ee ee ee ee eee 
4.00 +54.00 +36 
+643 +3 
4.25 +60.43 +39 a 
002 Mi oe 3) 
4.50 te 20) feu eaten (+36) 
Se (+718) (+3) 
(4.75) (+74.43) (+39) 
(+757) (==3) 
(5.00) (+82.00) (+36) 
(+793) (+3) 
(5.25) (+89.93) (+39) 
(+832) (3) 
(5.50) (+98 .25) (+36) 
(+868) (+3) 
(5.75) (+106 .93) (+39) 
(+907) 
(6.00) (+116.00) 


Table 9.7 
% f(z) Af A’f 
3.00 32.00 
+19 
3.01 32.19 0 
+19 
3.02 32238 0 
+19 
3.03 32.57 0 
+19 
3.04 32.76 0 
+19 ae 
3.05 32.95 pete at) 
A eA ——~ (4-19) 
(3.06) (33.14) (0) 
(+19) 
(3.07) (33.33) (0) 
(+719) 
(3.08) (33.52) (0) 
(4-19) 
(3.09) (33.71) (0) 
(+19) 
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From the difference tables that have thus far been developed, it is 
evident that the first differences are determined from a consideration of 
the function values at two adjacent positions in the table. Second dif- 
ferences, on the other hand, are obtained by considering the differences 
between the first differences straddling the point in question, but these 
could also be written in terms of the function values themselves. 


Af(i-2-3) = Afie-s) — Afc-2) (9.8) 
or A*fiy.2-3) = (fs — fo) — (fe — fa) 
or A*fis-2-8) = fs — 2fe + fi (9:9) 


Second differences are therefore determined from a consideration of three 
consecutive function values. In a similar manner, it can be shown that 
the third differences are dependent on four consecutive function values. 
This is shown diagrammatically by the triangular wedge of Table 9.8. 
Regardless of the order, with the apex of the wedge containing the appro- 
priate difference in question, the range of influencing function values can 
be determined from a consideration of the type shown in the table. 


Table 9.8 
x f(x) Af A’ A®f 
Zi f(x) Difference in 


question 


Influencing } 23 
interval 


Ze f (Xe) 


Not all data are available in equal increments of the independent vari- 
able. This is especially true when tabulations are prepared from experi- 
mental results. For such cases a difference table that takes into account 
the magnitude of thé increment in x by dividing the difference in question 
by the appropriate influencing interval (see Table 9.8) is extremely useful. 


Such a table is called a_divided-difference table, and the differences are 
noted symbolically as Af, *f, ete. Table 9.9 is a divided-difference 
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Table 9.9 
a NV Ayo 


table for the example given previously as Table 9.4. The first divided 
difference enclosed by the dashed oval is obtained as shown in Eq. (9.10). 


Af - (312) (156) e156 
(CP10)2==(- 7) +3 

Similarly, the designated second divided difference is 
(82) = (ees?) 15 
(taLO) 544-5) +5 


Here, as in the preceding case, the second differences are constants and 
the table can be extended in a comparatively elementary fashion. 


= +52 (9.10) 


Mf = = 13 (9.11) 


9.2 Errors in Difference Tables 


If the error in question is an isolated misentry in a given function value, 
it is analytically possible to locate the error and to determine its value. 
Consider, for example, Table 9.10. The function in question is the same 
as that for which Table 9.4 was developed: however, for illustration, an 
error of +10 has been purposely introduced in entry f(8). Large devia- 
tions from the values that were determined in Table 9.4 are noted in the 
region of the table enclosed within the dotted lines. Moreover, it is 
observed that as the order of the difference increases, the deviations from 
previously calculated values alternate in sign. This error is propagated in 


the manner that would be expected from a single error source e, as demon- 
strated in Table 9.11. Only the error parts of the table are shown. 


196 modern methods of engineering computation 


Table 9.10 
8 eS ee eee eee 
© f(z) Af A?*f A‘*f A‘f 
Ne ee eee 
3 +32 
+22 
4 +04 +6 
+28 0 
5 +82 +6 ae 
+34 = eo ae 
6 +116 _— bere a oe ; +10 
— 40 a aid. 
7 Ls Ase) See ae +16 —40 
nt +56 —30 
8 | +212 | —14 +60 
oe +42 +30 
—o LOZ = +254) ae te +16 ==40 
2 09) ee 
10 +312 Cr eS +10 
+64 OTe 
i +376 +6 0 
+70 0 
12 +446 +6 
+76 


By comparing the fourth-difference columns of Tables 9.10 and 9.11, 
and assuming that the value of the fourth differences should be zero, the 
error can be determined from any of the following equations: 


“6 — 110 
—4e = —40 (9.12) 
+6 = +60 


The error-propagation table for two adjacent misentries, assuming that 
both errors are equal and of the same sign, is given as Table 9.12. Other 
error-propagation tables could be developed in a similar manner. 

For the single-misentry case, illustrated in Tables 9.10 and 9.11, the 
largest deviations from expected values, for the even-numbered differ- 
ences (that is, A?f, A4f, etc.), are contained in the same line as the initial- 
function-value error. “This information is helpful in locating the more 
probable error sources in the given table. For two adjacent misentries, 
shown in Table 9.12, the location of the error source may not be as 
obvious. This is particularly true when entries have been rounded off. 
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ve Gy Table 9.11 


Li+41 0 SS ae +e —Ae 
(oa = 
i € 
Li+2 0 0 ae ag 
= 
0 0 ln 
Ti+3 0 0 0 
Table 9.12 
z f(z) Af A’f IND Af 
ae 0 De eae: 
wee 
0 St +e 
Bios 0 ee +e —3e 
ae te —2e 
Ties +e —eE +2e C 
0 0 
Lie +e —eE +2. C 
Iga =6 +2¢ 
Vi4l 0 Se +e —3e 
0 ae one: 
Vi+2 0 0 aie ns oe +e 
0 Oe 
Li+3 0 0 0 


9.3 The Difference Operators A, V, and 5 


To be able to develop compact, analytical expressions for the various 
interrelationships that exist between the entries in a difference table, and 
to use these in the development of interpolation and extrapolation for- 
mulas, it is first necessary to define the symbolic linear operators A, V, 
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and 6. By definition: 


forward-difference operator 
backward-difference operator 
central-difference operator 


E = SHIFT OPERATOR 


The shift operator, denoted by the letter H, is also of importance. 

It should be understood at the outset that the introduction of these 
symbolic operators will not alter the magnitudes of the various entries in 
a given difference table. Their purpose is to facilitate understanding of 
the basic concepts and to provide a more usable form for the presentation 
of results. 

In the work that will be discussed in this section, it has been assumed 
that the independent variable x is subdivided into equally spaced intervals 
of magnitude h. It has also been presumed that as x increases there will 
be a positive increase in the function y = f(x) and all its differences A’f. 


| The forward-difference operator A assumes that the difference in question 
is referred_to the least of the considered xz positions.. That is, 


A*f(z) = A*™ f(x + h) — A™-}f(z) (9.13) 


For the first difference, referred to position 0, 


Afo = fi = fo (9.14) 
Similarly, 


A*fo = Afi — Afo = (fe — fr) — (fi — fo) 
or A*fy = fe — 2f1 + fo (9.15) 


In like manner, 


A*fy = A®fi — A*fo = (fs — 2fe + fi) — fe — 2f1 + fo) 


or A®fo = fs os 3fe =— afi az fo (9.16) 
The third forward difference referred to position 3 is given in Eq. (9.17). 
A*fs = Af, — A*fs = fe — 3fs + 3f1 — fs (9.17) 


In all these cases it is to be noted that the difference referred to a 
particular position contains the function values at that position and at 
those forward of it; thus the notation forward differences and forward- 
difference.operator. A difference table in symbolic forward differences is 
given as Table 9.13. A forward-sloping line, to emphasize the forward 
differences referred to the 0 position, is shown in the table. 

The shift operator E is used to define the value of the function at posi- 
tion x + h, that is, f(@ + h), in terms of f(x); 


fle + h) = Ef(z) (9.18) 
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Table 9.13 


en ee ee 
xr f(x) Af A’f A‘f Af 


Since these function values are also related through forward differences, 
it should be possible to determine the relationship that exists between A 
and H. For illustration, consider the first difference referred to posi- 
tion x. 


Afe = fein — fe = Efe — fe = (E — Mfa 

The relation between the operators is therefore 

A=EH-—1 

(9.19) 
or E=1-+A 
For second-order difference 
Ache = Seton ae 74 Pa + f: = (E ad Dia, + fr 

or Ay, = (fF — 1)7f, (9.20) 


By extending this concept, it can be shown that the forward-difference 
operator A and the shift operator H are related according to Kq. (9.21). 


A’fs a (E a 1) "fz 
An = (E—1)" 


The backward-difference operator V assumes that the difference in ques- 
tion is referred to the greater of the considered_z positions. That is, 
Vote = Ws 5 V"—"fe-n) (9.22) 


By noting from Table 9.8 the interval of function values which influences 
a particular order difference, it is evident that backward differences will 


V$x= Ve e0:) 


(9.21) 
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contain the function value of the point in question and the values of those 
points backward from it. A difference table in symbolic backward dif- 
ferences is given as Table 9.14. As in the preceding forward-difference 


Table 9.14 
z f(z) Vii Vf Vif V7, 

—3 Y-3 V*y_2 Vty_1 
Vy-2 V3y_1 

—2 Y-2 

— 1 Y-1 

0 Yo 

VYy41 V3y42 

=p Y+1 V7y 42 Viyss 
VY +2 V3y43 

+2 Y42 Vy 4s 
Vy +3 

+3 Y+3 


case, a sloping line has been shown to emphasize the reference locations. 
The 0 position was chosen for illustration. 
By noting that 
1 
f(z) = Eff@—h) or fle —h) = = f@) (9.23) 


the first backward difference can be written in terms of the shift operator. 


Via = fe es = Ve — ale 


Therefore, 
af.= (1-5) k= - 2), (9.24) 
or V = (1 -E"}) (9.25) 
This, in turn, can be related to the forward-difference operator. 
Ve Z and A= u (9.26) 
1+ A eS hy 


It would seem, from the two cases just considered, that the linear oper- 
ators A, V, and E behave in the same manner as algebraic symbols. This, 
in fact, is the case, and it can be shown that all the operators A, V, 6, and 
E satisfy the distributive, associative, and commutative laws of algebra. 
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The central-difference operator 6 is used when the reference locations are 


contained in a horizontal line in the difference table. A difference table 
in symbolic central differences is given as Table 9.15. With such a Sys- 
tem, central-difference expressions will contain the same number of func- 
tion values forward of the reference location in question as backward from 
it. 


Table 9.16 
eee 
z f(a) of of def é‘f 
ee a 
==} Y-3 
5-54 
—2 Yy—2 6_2? 
6_34 6_343 
=] Y-1 627 6 13 
b_16 6_13 
0 Yo 50? 594 
419 54348 
+1 Y+1 641? 6417 
5436 84349 
+2 Y+2 542” 
5454 
+3 Y+3 


In central differences 
af (2 +5) = fle +m) — SG) = vf) 


or bE“f(x) = (FE — 1)f(2) (9.27) 
This gives 

6 = Et4 — K-% (9.28) 
By substituting the value of # from Eq. (9.19), 6 can be expressed in 
terms of A. 

6 = A(1+ A)” (9.29) 
Another useful relationship can be established by multiplying both sides 
of Eq. (9.28) by +t”. 

674A = HE-1=A (9.30) 

By squaring both sides of this equation, it is seen that the second-dif- 
ference operators 6? and A? are related through the shift operator L. 


6H = A? (9.31) 


For example, the second central difference referred to location 0 equals 
the second forward difference referred to location —1. 
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6?(Hy_1) 
or OU ed (9.32) 


I 


A’y_1 


A comparison of the appropriate entries in Tables 9.13 and 9.15 verifies 
this relationship. 

Table 9.16 is a summary of several of the relationships that exist 
between the linear operators A, V, 6, and #. 


Table 9.16 
A =V(l — Vv)" A=E-1 
y Seas . v= (1 — E“) 
8 = A(1 + A)-* 8 = V(1 — v)-# § = Et4 — E-% 
E=1+A E = (1 —v)7 


It is interesting to note that these linear operators can also be related 
to the operator D = d/dz in differential equations. If it is assumed that 
the function f(x) in question is single-valued and continuous in the range 
in question, expansion in a Taylor series about point x yields Eq. (9.33). 


fet) =f@+tar@tir@+--- 


h2D? 
2! 


a EfG@) = fa) = (1 pe Ee ) fz) (9.33) 


The term in the parentheses, however, represents the series e*?. 
Therefore, in symbolic form 


oad ua (9.34) 


Equation (9.34) is the fundamental linking relationship between numeri- 
cal differencing and the formal analytical processes of differentiating and 
integration. The interrelationship can be extended to other linear opera- 
tors considered in this section, which are shown symbolically in Eqs. (9.35). 


E = e&P 

Ae =e al 

Tee (9.35) 
6 = 2 sinh 44hD 


YENI VED LMR NETFIRMS TIN 
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The operators A and V can also be related, in expanded form, to the 
operator D. Given the basic symbolic relationship, repeated here as Eq. 
(9.36), 

A= ¢! — 1 (9.36) 
series expansion will yield 
h2D2 h3D3 hiD4 


A=hD+ ap ease Ree (9.37) 


The second forward difference could be obtained in a similar manner. 
A? = (eD — 1)? = e*D OD + 1, =-.-- (9.38) 


Alternatively, by recognizing that linear operators can be treated the same 
as algebraic quantities, A? can be obtained directly from Eq. (9.37). 


h2D2 h3 D3 htD4 2 
at = (1D + a1 Bl re) 
= WD? + h83D§ + UY%{ohtDi +--+ (9.39) 


Expressions can be obtained for higher-order forward differences as well 
as for backward differences in a similar manner. A summary of several 
of these is given in Table 9.17. The inverse system can also be defined 
by series expansion, and these are summarized in Table 9.18. 


Table 9.17 

h2.D2 h3 D3 hiD* h2D?2 he D3 hiD4 
eee 5 3! 4! eal)“ omenlaeatssint 
A? = h2D2 + hi D3 — 74 gh*D* -- See bei v2 == h2D2 — h3 D3 + 14 oh4D* = FO 
A3 = h®D? 4+ 36h!D4 + 54h5D5 + - - - V3 = h3D3 — 36h4D4 + 54h5D> — - - - 
Table 9.18 oa g 2D Back WRRY 


vy vs Vv 
AS sires ara pe ) 


Ay A-2z Le 


poe 
ah 


1 
Dt = = (at — At + 144208 Di = SE + VI + 1a¥! 


— Hart) ++ >) 
1 
Di=L(ar—sattyat—--) | D = + uvit vet) 
1 
ee As DATA IGAS De -) Dena cae, tan eM ch, nes ) 
h* 


Gop Be assy cASE WHEN A'S CONSTAAT 
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It should be understood that differential or difference operators of the 
type given in Table 9.18 can equally well be listed in terms of the func- 
tion values themselves. For example, from Eqs. (9.14) to (9.16), 


Afo = fi — fo 
A*fy = fo — 2f1 + fo (9.40) 
A®fy = fz — 3f2 + 3fi — fo 


Therefore, since 


ih 
Dfo = 5 (Afo — }4A%fo + ¥6A%o— -- 


Oe JG = Uh = ie 2 ea) 
+ 146(f3 — 3fe + 3f1 —fo) — - - '] 


which gives 


Dfo = = (-1Mfo + 18h — Of + 2fe— -- >) (9.41) 


9.4 Interpolation 


As pointed out in Sec. 9.1, one of the most often encountered problems 
associated with the use of tabulated quantities is that of interpolation. 
This is especially true when the function values are dependent on more 
than one variable. In this section, however, attention is directed toward 
the problem of interpolation in single-independent-variable systems, even 
though certain of the methods could equally well be extended to the more 
involved cases. 

The basic problem in question is the development of numerical meth- 
ods for calculating values of a function corresponding to values of the 
argument that he between tabulated points. The most elementary form 
of such interpolation is referred to as linear interpolation or taking pro- 
portional parts. This assumes that in the range in question the function 
can be adequately represented by a linear function. As demonstrated 
later in this section, this is but a special, limiting case of the more general 
interpolation problem. 

There are many approaches to the development of interpolation for- 
mulas, and several of these are considered in this section. However, con- 
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sider first the ‘‘difference-table behavior’ of certain known analytical 
relationships. 
1. y ts a linear function in x. 
Y = Mr + Ao (9.42) 


If the differencing interval is assumed to be h,|the first forward dif- 
ference 1s that given by Eq. (9.43). 


Ay = f(x +h) — f(z) = (ai)(u +h) + ao + (airz — ao) = anh 
(9.43) 


Noting that the second and higher derivatives of y with respect to x 
are zero makes it possible to obtain this same relationship from the 
equations of Table 9.17: 


Ay = hD(y) +0 =A < (ayx + ao) = hay (9.44) 


Equation (9.45) is also evident. 
AN =) fOr ae 2 (9.45) 


2. y ts a quadratic in x. 
yY = age? + ax + ay (9.46) 


Using the equations of Table 9.17, 
Ay = h?(a2) + h(2aex + ar) 
A’y = h?(2a2) (9.47) 
Ary = 0 for tee 3 
Ba tsa cubic ne: 

Y Ae? a Get? SP aie ae (9.48) 

Ay = h3(a3) + h?(8asx + ade) + A(3a3x? + Zax + ay) 

A*’y = h3(6a3) + h?(6as% + 2a2) 

A*’y = h3(6as) 

A*y = 0 forn = 4 


(9.49) 


From these three examples and Table 9.17, it is evident that |the nth 
difference of a polynomial of order 7 is a constant. \ Moreover, the value 


of that difference is ( 
i Arty = h™Dry = a,hn! = Vy (9.50) 


The values of the (7 + 1)st and all higher differences are zero. 


Diy= a, oY) 
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In view of the above, if the nth difference in a “regular” difference 
table is a constant, it is reasonable to assume that an nth-order polyno- 
mial can be made to represent the function in question, and this is one of 
the approaches used in developing interpolation formulas. It is to be 
noted, however, that there are certain analytic functions for which this 
assumption is not true. For example, should there be contained within 
the interval in question singularities or severe fluctuations, matching of 
the interpolating polynomial and the function at equally spaced ‘‘pivotal 
points” is not sufficient. 

Figure 9.1 illustrates the general interpolation problem in question and 
defines the notation that is used. It is assumed that the function values 
are known at equally spaced increments of the argument and that a dif- 
ference table either is available or can be defined. The problem_is to 
determine the function value at « + 6h. Itis presumed tha 40 < 0-< 1.0..4 

Interpolation formulas are first developed, using the operator notation 
of Sec. 9.3. In terms of forward differences, the function value at x + 6h 
is 


f(a + 0h) = E*f(x) = (1 + A)*f(@) (9.51) 


Expanding the term in parentheses by the binomial theorem yields 
Newton’s forward-difference interpolation formula: 


eter = [14 a+ Dar 


6(@ — 1)(@ — 2) -- + (@ —n +1) 
n! 


+ 


ar | f(z) (9.52) 


To illustrate the use of Eq. (9.52), assume that difference table 9.19 is 
given and that it is desired to evaluate the function at r = 6.5. (Note 
that Table 9.19 is a part of difference table 9.4.) Again, for emphasis 
a forward-sloping reference line has been drawn. To the right has also 
been included a symbolic forward-difference table. 


Figure 9.1 
Desired function value 
$23 AGS) G 
| | | 
nee eee, 
| ye A a \ | 
* | LS senisen | I 
1 x \ sm \ 
| | |on| | | 
Me Bd eh 
fh | | H [ sll x 
SEW) Gy) Be X+h x+2h x+3h 
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For the case considered, h = 1.0 and the 0 to be used in Eq. (9.52) is 
0.5. Evaluation then proceeds as follows: 


f(6.5) = f(6.0 + 6h) = f[6.0 + (0.5) (1.0)] 


2 E + Ae ea a © | 706) 


Sey oA, 


6=41) 
oe =i 


(0.5)(0.5 — 1 


+116 + (0.5)(+40) + 5 a (+6) 


+135.25 (9.53) 


The analytical function corresponding to Table 9.19 is f(x) = 3x27 + a + 2. 
Substituting x = 6.5 in this equation results in the same value for (6.5) 
as that given by Eq. (9.53). 

In the just preceding example the value of the argument, that is, 
x = 6.5, was near the beginning of the difference table, and consequently 
all the forward differences referred to c = 6 were defined. Consider now 
the situation where evaluation is desired for = 9.5 and it is assumed 
that only the difference table given in 9.19 is available. Referring to the 
symbolic forward-difference table of 9.19 reveals that only f(9) and Ay are 
available for use in Eq. (9.52). For this and other cases where values 
are desired near the end of a difference table, Newton’s backward-difference 
interpolation formula [Eq. (9.56)] should be used. From Table 9.16, 


Therefore, 


Heel Ob — a )jom (l= Vy (2) (9.55) 
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or 
f(x + 6h) =[itov+ CFU oe 4 ee? 

n (6 + 1)(@ + 2) = -(@+n-—1) ve | 70) (0.56) 
For « = 9.5, the reference point is x = 10.0 NY = —0.5. This gives 


fQ9.5) = f(10.0 — 0.5) 


Sle aL Oa 


(—0.5)(+0.5) 
2 


as f(10) i (—0.5)Vi0 <i 


= +312 — (0.5)(+58) + (+6) = +282.25 (9.57) 


If it is known that the difference table is “regular,”’ the reference point 
can be selected as x = 9 and 6 = +0.5 with the following result: 
f(9.5) = f(9.0 + 0.5) 


(0.5) (0.5 + 1.0) 
2 


= +254 + (0.5)(+52) + (+6) = +282.25 


(9.58) 


Even though this procedure results in exactly the same function value, it 
is to be recognized that this second case extrapolates beyond the range of 
influence of the differences used. It is therefore more subject to error and 
is not recommended for general use. 

Equations (9.52) and (9.56) are but two of the many interpolation for- 
mulas that can be and have been developed from a consideration of the 
relationships listed in Table 9.16. Table 9.20 is a summary of several of 
these. For completeness, Eqs. (9.52) and (9.56) have also been included. 

To illustrate the use of Gauss’, Everett’s, and Bessel’s central-difference 


interpolation formulas, consider the evaluation of f(8.5) for the difference 
table 9.19. 


1. Gauss’ forward-interpolation formula: 
f(8.5) = f(8.0 +=-0.5) 


(0.5) (—0.5) 


= +202 + (0.5)(+52) + : 


(+6): 4'0 = --227.25 


(9.59a) 


& = Amou NT OFF OF THE ) Ween 
VALUE « OF 6 =1,0 
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. Table 9.20 A= \weREMEVT 
K Newton’s Forward-difference Interpolation Formula : 
f(a + 6h) = f(x) + @Af(z) 4 A are aati 
55 He) me 9 one 
2 
VALU 
Newton’s Backward-difference Interpolation Formula ane 
oF 
ile + 0h) = fea) + ova) +P F 9 veya) + ‘Ae 
1 2) ena = il 
4 9 +6 + . Gee 


NS Fale 


Be Gauss’ Forward-inter polation Formula d. llustrated for Location O) 


rN 6 May Ley =| 
Wy 08 — I<? aes - 
f(z + 0h) = yo + 6 (644) + a (3?y0) + ——__-— Sea re 5 
a(@? —1)(8— 2), 0(6? — i ae. fe = 76 =r) 
is 4! ge et (2r)! 3 
& 6(6? = ie) Aeoeess (0? == r?) oe 
Bh y-2 ae sty sig 


vA Gauss’ gs gaeagaia PE Formula (Illustrated for Location O) 


wh 


By as ae. 
fle + 0h) = yo + doy-y)+ PEM ory)4 mee ¥) 
a(e? — 1%)(6 + 2) a(@ — ie 0 ee a) \ 
eee wt ent Hye Nes 
6(6? = 1”) a S60 (0? — 7”) ante 
Qr +1)! ar ae 


Boerait’s Interpolation Formula (Illustrated for Location O) 


9(@ — 1)(8 + 1) oy, ee ee eee, 


f(a + 0h) = eve a 


3! 5! 
o@—1)---@—r)(@+1)@4+2)--- @+r—1) ,, 
a a Qr+ 0)! ms] 
a(@ — 1)(@ —2 OO) Ome 
= E =o se ea 6?y0 + 5 
66 1)6= 2) oni = DOI 2 O47 =) “| 
e @r +1)! one 


a 


a 
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Table 9.20 (Continued) 


Bessel’s Interpolation Formula (Illustrated for Location O) 


Yoty: . 0(0 —1) f d’yo + &y1 
re +m = | ay ( : ) 
0(6 — 1)(6 — 2)(0 + 1) f dtyo + S41 
1 4! 2 
o(62 — 12) - - - [62 — (r — 1)3}(0 —r) [8yo + Ory 
cs (2r)! 2 
—1)(@-—% (6 — 1)(@ — 2)(@ — 34)(@ +1) ,, 
ae {« — 44) byy% + ee 5°yyg + Z Bl : 5°y ys 
= BS a eee 
eee a | 
2. Gauss’ backward-interpolation formula: 
f(8.5) = f(8.0 + 0.5) 
0.5)(1.5 
= +202 + (0.5)(+46) + ee (+6) + 0 = 4227.25 
(9.596) 


oo 


3. Everett’s interpolation formula: 


f(8.5) = f(8.0 + 0.5) 


(0.5) (—0.5) (1.5) 
6 


= | (.5)(4258) + +0)| 


(0.5) (—0.5)(—1.5) 
6 


= | (=0.5)(+202) a= +6) | = 4997.25 


(9.59c) 


_ 4. Bessel’s interpolation formula: 


f(8.5) = f(8.0 + 0.5) 


+202 + 254 (0.5)(—0.5) +6 + 6 
i‘ SS a y 2 
+ [(0)(+52) + 0] = +227.25 (9.59d) 


With regard to the choice of a particular interpolation formula, the 
central-difference formulas will, in general, result in faster convergence. 
Of those listed, Bessel’s and Everett’s are recommended. However, when 
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values near the end of a table are required, Newton’s formulas must be 
used. 

If the function represented by the difference table is a polynomial of 
degree m and the tabulated values are exact, the (n + 1)st and higher 
differences are zero, the interpolation formulas terminate, and interpola- 
tion is “exact.” For those other cases where the (n + 1)st difference 
oscillates about zero, the function numerically responds as if it were a 
polynomial of degree n. The more “‘direct’’ development of polynomial 
interpolation formulas to represent tabulated function values would there- 
fore seem in order. 

Consider, for example, the situation defined in Fig. 9.2. It is presumed 
that three equally spaced points —1, 0, and +1 have associated with 
them the three function values y_1, yo, and yi1. The origin is assumed 
to be located at position 0. It is desired to pass a single second-order 


polynomial of the type given by Eq. (9.60) through th hree poi 
: (9.60) 


y = ox? ft a,x a ao 


The three conditions 1S for a solution are given by Eq. (9.61). 
SOESTITUTING {yy = as(—h)? + ax(—h) + ao(1) 

day des fe yo = a2(0) +a(0) + ao(1) (9.61) 
= 5 0,1 Yar = G2(+h)? + aa(+h) + ao(1) 

These can be recast in the matrix form. 


healed ae Y-1 
0 0 a le faa 
eter |g Y41 


ly] (9.62) 


= 

° 
Lar} 

pS 

= 
5 

= 
I 


The unknown coefficients a; can be defined in terms of the known function 


Figure 9.2 
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| 
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values y; by calculating the inverse of [A]. That is, 


i 1 1 
a2 = 2h? a PA aN Pie 
[a] = [A“J[y] or a{=} 1 nh LY] yo (9.63) 
Qo 2h 2h | LY+1 


0 cee 0.4 


The corresponding central second-order interpolating polynomial is 
therefore 
Yor Yer 


ya — 2Yo T Y41 


Ua 


It is also possible to develop similar interpolating polynomials in terms 
of known function values forward of the point in question or backward 
from it. These are summarized in Table 9.21. "They could equally well 
be written in terms of the notation used in Table 9.20, that is, x/h = @. 
Consider, for example, the second-order forward interpolating polynomial. 


Yo — 2441 TF Yee Sue 2041 oe 


+2 
= x? Las 
y Dh? ah Yo 
Yo — 2Y41 + Ye —8Yyo + 4Y+1 — Yre 
Yy = 6? Sia 6 a Yo 
2 2 
Table 9.21 
Forward 
ee 
H A) Rasy V4.2 = Yo = 2Ys1 1 Yan 2 —3yo + 4y41 — Yue 
mie ! | 2h? atte 2h + ee 
{ 0) +f ,*2! 
ED afi aaah 
Backward 
ee re 
aa ame tbe BS X 
fe St AOL y—2 — 2y-1 + yo ye a yee 
| Tae Dh? ale oh z+ Yo 
=? =i 0) 
h h 
Central ind wa 
ye 1 2 ete yh 
| _ Y-1 — 440 7 Y41 SPY ape 
ans | YP Sa ee eee ee 
=| i 2) 4 +1 
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This can be rearranged as follows: 


(Yo — Y+1) — (Yai — Y42) 92 


— 


2 
4 ae oes 2041 us) + (=2yo + 2y+1) at yh 
A? A? 
= Eo + Aye + ye 
a(o <1 
or Y = Yo +0 Ayo = A*Yyo (9.65) 


which corresponds to a terminated Newton’s forward-difference interpola- 
tion formula. In similar manner, the central interpolating polynomial of 
Table 9.21 reduces to a terminated Gauss’ (central-difference) backward- 
interpolation formula. == =e 
———"While the arrangement of the various terms in n the antorpolntion ¢ equa- 
tions_of Table 9.21 is logical and consistent with the assumed polynomial 
(9.60), “Other forms could also be written. For example, in Table 9.22 
each of the bracketed terms is a separate polynomial in x. Collection is 
with respect to the assumed- known function values y_2, y-1, Yo, etc. The 
equations given in Table 9.22 suggest an interpolating polynomial made 
up of linear combinations of 7 + 1 cepa polynomials in z. That is, 


1=0 
Table 9.22 
Forward ; 
x " shee ea 
Z (h) (2h) - (ee a pen 
(z)(z — h) 
0 ——— EEE 
at (h) (2h) | Y+2 
Central — 
_ | @Xz =) + [SPE 
as Pie (h)(—h) 
SS 
-} Ee a (x)(@ + h) 
ae Gan |? 
Backward — ¥ 
_[@e +h) eh ote va 
S te Peer |e Hen |= 
}____+_ ____=_| 
=O +1 


(c + h)(x + 2h) 
+| (h) (2h) |» 
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where the [L;(zx)]’s are nth-degree polynomials in the independent vari- 
able x and the summation includes a total of n + 1 terms. It is to be 
noted that these need not be referred specifically to the desi 
0,1,2,...,n. They could apply equally to the points/—3, —2, —1, 
0,+1,...,n — 3,orany othern + 1 consecutive points. 

In developing expressions for interpolating polyno 
given in Eq. (9.66), the more general approach of Lagrange is here used. 
That is, it will be presumed that the function valugs are not necessarily 
given at equal spacing of the independent variablé x. Moreover, it is 
assumed that the individual polynomials are identically zero at all pivotal 
points other than 7. For that one particular Jocation, they equal +1. 
These conditions are defined by de 


(0 eaten 

[L;(2;)] aa | ae for j ay 

From the first of these, by recognizi @ that each zero represents a root 
of the polynomial in question, Eq. (968) can be written immediately: 


(9.67) 


La(a,) = Aj(w — 20)(@ Yom)» < + fe — m4) (0 — taxa) + + + (@ — a) 
\ (9.68) 


[It should be noted that the terny (x — x;) is purposely missing from (9.68).] 
Applying the second of Eqs. (9.67), 


1 = A;(x; — 20) (2; a WR ey ta si a) ae ee) 
hae Fie 1 
oa %0) (%j—/at1) . (<j — Xj-1) (aj — ai41) + * * (Bj — In) 
/ (9.69) 


jf 


The individual TON are therefore defined by 


_ @— a —m) +++ &— ae — a) + me) 
L,(x;) = : 
ay) (xj — Ae, — Ra) "(DAS Baa) el en eed 


/ (9.70) 
In expanded form, these are 
/ 
/ (ols Ti} Cees) a a 
/ Lo = ; 
. - (to — %1)(%o — @2) (ao — Xn) 
N (= Xo) (= .22) a Sa) 
/ Ty = 
/ (x) (t1 — 20)(%1 — a2) - - \(x1 ee (9.71) 
ry Oe eee 


(C3 — 20) Gre (Oe ~ Tet) 
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~ 


Lagrange’ s interpolation formula is frequently written in the ee 
notation of Eq. (9.72). ve 


J (9.72) 


: 


(41 — 2o)(4K— to) + °° (41 — 2y _ 7 

(x — 2o)(% \ 21)(4 — 43) + - - (x L xp) 

(2 — Xo) (X2 BIS Ce ts) ete 2) 
Ye = aoe = a)/- + - (= 24) 


BG Lo) Gn — %) eRe (ee Lie) 


Voge 2° * 


If Eq. (9.73) is applied to\ the eqyal-spacing second-order central- 
difference case of Table 9.22, thé interpolating formula will be developed 
as follows, starting with the Be ae given in Table 9.23: 


/ 


Table 9.23/ \ 


\ 
Vi Second-order 

Notation of Equation (9.70) central-difference notation 

/ 


/ Xo Z1 (where x1 = —h) 
ys zy xo (where zx» = 0) 
/ Xe 241 (where x41 = +h) 


£ Caen if (x — 
a (a 2) (1 — x) a (a = t1)\to = 21) 
J, @ = 2a)@ = a) 


J > eat — 2) * 


__@ Je) (@ + h)\(@—h 
~ (hE O)(=h = 2) 2" OF HO - i) 
/ (x + h)(z — 0) 


/ Uy (eiea et Waa 
(&)( — h) (@@—h(eth) (le +h) 


Cea Meenas fame 


or y= (9.74) 
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It is to\be noted that this is the same equation listed in Table 9.22. It 
should also be observed that in the development of Eqs. (9.71) the coef- 
ficients of the y’s depend not on the specific values of x chosen, but only 
on the interval of spacing. The corresponding fourth-order central- 
difference interne ane polynomial is given as Eq. (9.75). 


_ (x) (@ = h(a \ 2h) (a + uh ene = bya 2h) zh 
24h Ss —6h4 as 
age —h)(«—- 2h) (x. + h)(a + 2h) epee —2h)(x + h)(x + ZL 
4} Oe n —6h! 7) 
\ 28 (x — h)(y + h)(@ + 2h) 
“ere 24h! Yy+2 (9.75) 


To illustrate the use of the general Lagrange interpolation formula [Eq. 
(9.73)], consider the evaluation of y = e*/for = 0.88372, given the tab- 
ulated function values of Table 9.24. It is to be noted that only three 


\ 
\ 
\ 

X 


Table 9.24 
a y =e \ 
0.85 2.3396 
0.88 2.4109 
0.90 2.4596 


values have been given, and these are not equally\spaced. The interpola- 
tion equation corresponding to Eq. (9.73) is then\ 
\ 


_ (0.88372 — 0.88000) (0.88372 — 0.90000) 
i (0.85 — 0.88) (0.85 — 0.90) 
(0.88372 — 0.85000) (0.88372 — 0. 000 re(r 
(0.88 — 0.85) (0.88 — 0.90) 
(0.88372 — 0.85000) (0.88372 — 0.88000) \ 
(0.90 — 0.85) (0.90 — 0.88) 


© 3398) 


@. 4596) (9.76) — 


or 


y = — (0.04037) (2.3396) + (0.91493) (2.4109) + (0.12544) (24596) 
= +2.4199 (9.77) 


To the four decimal places given, the answer is correct. 
One of the advantages of using the Lagrange interpolation equation is 
that a difference table need not be constructed. Less computer storage 
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is therefore required. It is to be noted, however, that as the procedure 
requires the subtraction of almost equal numbers, a large number Of sig- 
nificant figures must be carried in the calculation to obtain the/desired 
accuracy. A second shortcoming is that it is impossible to know at the 
outset the number of points needed for a particular accuracy. When 
points are added to improve the accuracy, all polynomials must be 
recalculated. 

With regard to the general question of accuracy, errorg in interpolation 
result from several sources. Truncation errors, for example, occur when 
terminated rather than the infinite series are used. The errors associated 
with such procedures correspond to the sums of the remaining terms of 
the infinite series. It is to be noted, however, that no general rule can 
be formulated for the definition of these remainders unless further knowl- 
edge of the tabulated functions is available. / For that particular case 
where the nth difference terms are included in the interpolation, the 
(n + 1)st differences\are constants, and the (n + 2)nd and higher dif- 
ferences are zero, the remainder can be defined. For Newton’s forward- 
difference interpolation formula it is given by 


hrtiflntti () 
in + 1) 


where f!"+11(£) is the value\of the (n + 1)st derivative of the function, 
assumed to be constant over/the range % <&<2,. For Newiton’s 
backward-difference interpolation formula, the corresponding remainder 
is 


PS HOLL NO 2a Sn ey (9.78) 


iN 


hr+1fint11(£) 
ewe apts 9 fh3" 9.79 
R= mtd! ae KDE +2) 20's nm) (9.79) 
For the Lagrange formula, \ 


\ 
\ 


(2 —foo)(e — a1)(w — 22) \, + @ — te) 


ose Cary 


pt ej 2-30) 


When the spacing is equal, Eq. (9.80) reduces to Eq. (9.78), the Newton 
forward-difference interpolation formula. Remainder terms for these and 
the other interpolation formulas are listed in most texts on numerical 
analysis. \ 

The rounding off of numbers.is another source of error. Should this 
occur in the values of the given tabulated function values; there will result 
oscillations i in the (n + 1)st differences of the difference table. This was 
demonstrated i in Table 9.5. Rounding off in the individual terms of the 
interpolation formulas and in their sums will also contribute to error. 
This is especially true when the Lagrange formula is used. 
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9.5 Inverse Interpolation 


Interpolation, the subject of Sec. 9.4, requires the ae of f(z) 
for a particular xz located between two tabulated values. 
polatio! nh is concerned with the problem of given f(x), determine z. 

Since ¥ r all but the most elementary situations the interval of the 
tabulated values of y = f(x) is not constant, Lagrange’s interpolation 
formula for tnequal spacings can be directly yas . 


pa CL = yy ys) a) e a 
(yo — ys) (yo 12) eee — Ya) : / 
YY z= We) (y — Y2) ele Ok 
(Ys — yo) (yr — y2) > °° XY — Yn) 
(y — yy yy — ys) + KY Wn ait 
acy = yo)(Y2 —\ys)(y2 = Ys) FS em Oe (9.81) 


For those cases where the function’ Sain f(x) are tabulated at equal 
increments of the argument x, an alternative, iterative procedure can be 
developed directly from the equations of Table 9.20. Consider, for illus- 
tration, the first of the es formulas listed, Newton’s forward- 
difference interpolation formu hich is here repeated: 


jee + 00) = $2) + astey/s OXY aytey + 


be ere: 
“i ve ve A*f(x) (9.82) 


It will be assumed that 0<6<-+1 eodhiine the x position in Eq. (9.82) 
corresponds to the A) location in the difference table. That is, 


fa)-= yo A*f(x) = Atyg 
/ Af(x) = Ayo (9.83) 
A A*f(xz) = A%yo \ 


In Bae the known function f(z + oe will bé denoted as ye. Equa- 


(9 — 1) 


w= w/t oan ae iain Yd oe 
6(6 (6 oy 
ras )( ub (9.84) 
/ 6(9 =a 
wet yo = 8Aye + OS ary t - = 
OCG = 1) (0 —-D) cca; 
yak ) A”yy (9.85) 


n! 
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A first approximation to 6 can be obtained by terminating (or truncating) 
the series after the first difference Ayp. 


Ya — Yo = 10 Ayo or 19 = eeeaue (9.86) 
Ayo 
By using\ this computed first approximation to 6, a second approximation 


can be obtained by including two terms of the series’as indicated in Eq. 
(9.87). 


DGera 
Ye — Yo = 28 AYo =P a A*yo (9.87) 


\ 


56 19(10 —/1) 


Ne & oe aE Ae} 
a. (Ye Yo) D Yo (9.88) 


or og 


AYo 


The process is repeated according to Eqs. (9.89) until the change in 6 
between two successive iterations is equal to or less than the desired error. 


(2.86 — 1 O(o8 — 1)(.8 — 2 
Yo — Yo = 30 Ayo + ae A*yo + ie ae one 
50(s8 — 1) s0(38 — 1)(s0 — 2) 
Yo — Yo = 40 Ayo + Spat iat! A’yo + 31 A - (9.89) 
4 1968 DONS 260 — 8) 44, 


4!\ 


VA ' : : 
As an example of inverse interpolation, consider the determination of 
x for f(x) = 24,200, giyen the difference Table 9.25. In terms of the nota- 
/ 


Table 9.25 


/ 89 24,351 \ 3 
90 24,596 2 


91 24,843 
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tion of Eqs. (9.85) through (9.89), 


Ye = 24,200 Yo = 24,109 (corresponding to x = 88) 
Ayo = 242 y, 
At = 3 ie “ (9.90) 
A’y) = —1 
ye — Yo = 91 
From Eq. (9.86), 


From Eq. (9.87), : | 
(0.376)(0.376 — 1.000) 


91 = ,6(242) + : (3) 
or 26 = 0.378 aa (9.92) 
Using the first of Eqs. (9.89); | | 
0.378) (078 — 1.000) \ 
o1 =) 6043) ety = ) (3), 


\ 


< 
a (0.378) (0.378 — 1\000) (0.378 — 2.000) 


; (1) 
ys \ 
which gives / 
of 30 = 0.378 (9.93) 
The valtie of x corresponding to f(x) = 24,200 is therefore 
ay + 6h = 88 + (0.378)(1) = 88.378 (9.94) 


9.6 Numerical Differentiation and Integration 


It has been shown in the preceding sections that continuous “‘well-ordered”’ 
functions can be conveniently treated even though they may be repre- 
sented only in tabular form. With appropriate formulas, interpolation, 
extrapolation, and inverse interpolation can be carried out to any desired 
degree of accuracy. In reality, then, the formulas developed in Secs. 9.3 
to 9.5 can be considered to actually represent the real f(x), at least over 
a limited range of x. 

Interpolation and extrapolation are not the only numerical processes 
required in engineering and scientific analysis. Differentiation and inte- 
gration must also be performed. If the function in question is described 
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only in tables, the process by which the value of the derivative of the 
function at a given location is determined is known as numerical differ- 
entiation. Similarly, by numerical integration, the value of the integral 
Over a specified interval can be determined, even toguEh the function is 
defined only at discrete points. 

An obvious method for obtaining numerical approximations to the 
value of the derivative is to differentiate the interpolation formulas. As 
stated above, these represent to the desired accuracy the real f(z). 
Numerical-integration formulas would be obtained by integrating these 
same interpolation formulas. If over the range of x in question the equa- 
tions adequately represent the function values at the tabulated points, 
the developed numerical-differentiation and numerical-integration formu- 
las should provide reasonable approximations to the desired functions. 

In general, the process of numerical integration is stable and depend- 
able. Such is not always the case for numerical differentiation. The sit- 
uation is best illustrated by considering Fig. 9.3. Here it is assumed that 
a given function f(x) is prescribed over an interval a<2z<b. More- 
over, values of the function are presumed to be specified at a, at b, and 
also at one intermediate point. In addition, there is shown on the figure 
an approximating polynomial, an interpolation equation, which passes 
through these same three given points. It is to be noted that the approx- 
imation underestimates the true value of the function in part of the range 
in question, and overestimates it in other parts. This is typical of the 
actual situation that is observed when approximating polynomials are 
used to represent given functions. Since i. : f(x) dx is equal to the area 
under f(z) between a and 6, and since the approximating polynomial both 
underestimates and overestimates, the errors in numerical integration tend 
to be compensating in nature. Truncation is therefore less of a problem. 
Numerical differentiation realizes no such benefit. It is obvious that the 
derivative, that is, the slope of the curve, of the approximating polyno- 


Figure 9.3 


Approximating 
polynomial 


Given f(x) 
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mial may be markedly different from that of the given f(x). This is 
especially true when f(x) varies appreciably within the interval in ques- 
tion. Two methods can be used to improve accuracy in numerical dif- 
ferentiation. A higher-order approximating polynomial, one that requires 
a larger number of given pivotal points, can be introduced. The second 
method requires that the interval in z between the given pivotal points 
be reduced. 


9.6.1 Difference Expressions 


In Sec. 9.3 it was demonstrated that there exist relationships between the 
operator D (where D = d/dz) and the difference operators Aand V. The 
specific relationships were listed in Tables 9.17 and 9.18. Similar expres- 
sions also could be developed relating D and 6. The important point is 
that the operator D is presumed to represent the derivative in a con- 
tinuous system, whereas A, V, and 6 correspond to values tabulated at a 
number of separate and distinct points. 

It has also been demonstrated [see Eq. (9.41)] that the operator D can 
be expressed in terms of function values at known points. The exact 
relationship depends on the order of the approximation and the particular 
selection of interpolation formula. 

A second-order approximating function, one where A’, V?, and 6? are 
constants and higher-order differences are presumed to be zero, can be 
obtained by passing through three consecutive points a single second- 
order polynomial of the type given in Eq. (9.60), here repeated as Eq. 
(9.95). 


y= ar’? + ayxr+ a ~ (9.95) 
For symmetrically placed pivotal points, as indicated in Fig. 9.4, 
ao = Yo 
Y4+1 — Y-1 
ay = ——_ 
2h (9.96) 
2 ga = or Vas 
ag = 
2h 


This gives, for the difference operator evaluated at x = 0, 


D(Yo) = (2ax% + ay)220 = Ar 


Y+i — Y-1 
Os ~ D(yo) = ys (9.97) 


In like manner 


D*(yo) — Y-1 aie 2Yo ele Y+1 


re (9.98) 
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Figure 9.4 


The corresponding second-order forward-difference expressions are 


Ue 1 ays Tye 
2h 


D(Yyo) (9.99) 


Yo > 2Y41 + Yre 
h? 


D*(Yo) (9.100) 


It should be noted that in both cases the second-difference expressions, 
Eqs. (9.98) and (9.100), are of exactly the same form. This is not the 
case for the first differences. 

To illustrate the development of “‘higher-order’’ difference expressions, 
consider the fourth-order case with assumed central placement of the coor- 
dinate reference. (See Fig. 9.5.) The selected polynomial is 


y = ase* + azz? + aox? + aix + ao (9.101) 


Proceeding as before to evaluate a4, a3, @2, a1, and dp» in terms of the 
assumed, known function values at locations —2, —1, 0, +1, and +2 


Figure 9.5 
! | ; 
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yields the five linear equations (9.102): 


Y_-2 +16h4 —8h? +4h?. —2h +1] a, 
Wei +h4 —hi +h? —h +1]! as 
yo |= 0 0 0 OM HET | loa: (9.102) 
Yaa cmc Meni ae wei Meeks ela 
Y42 +16h4 +8h9 --4A2 +9n “EV | |i, 


The values of the constants become 


“yee —4y a Oe — Sy er ee 


a 24h! 
ted + 2y-1 — 20e1 4 Yy2 

Poi a: : 12h3 

1 —<y_»  16y_.—20ye 4 10yan — (9.103) 
aC = 2 1 0 +1 +2 
N\e ae Dah? 
eee HYya2 — 8Y-1 + OUsia ee 
12h 

ao = Yo 


With these, the derivatives evaluated at x = 0 can be readily defined. 


Yo = Ao 
Yo = D(yo) = a4 
Yo = D*(yo) = 2az (9.104) 
Yo = D%(yo) = bas 


Yo = D4(yo) = 24a, 


Substituting the values of the constants into Eqs. (9.104) yields difference 
expressions for the first four derivatives. 


Ys SYSt st Sy as 


D(yo) = 12h 
—y_2 + 16y_1 — 30 16y41 — 
Digte= ga a ot 16yy1 = 9e9 
49) 5 (9.105) 
D?(yo) = Y-2 Vie) > PAs + Y+2 


an 


Dy) = +Yy-2 — 4y-1 = = Aiea a Uae 


It is evident that equations similar to these can equally well be developed 
in terms of either forward or backward differences. 
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As was true in the development of interpolation formulas, it is desirable 
to have an indication of the magnitude of the error associated with a 
given assumed polynomial approximation. Several methods for doing 
this are available; the one selected for illustration presupposes the expan- 
sion in a Taylor series of the assumed continuous function f(x). That is, 
f(z + h) = f(z) or + Dw) +5 DX) +5 “ Dity) +5 Dy) aaa 

(9.106) 

Expanding about point zero, 


Y41 = Yo + h D(yo) T she D°(Yo) f 3 = She D*(Yo) 4 a = Dye 


(9.107) 
Similarly, 


1 i il 
ge) D9) eS kD ya) — 3% Pwo) eer lee 
(9.108) 
Adding corresponding members of Eqs. (9.107) and (9.108) yields 


Be Voi 240 aoe = h? D*(y) is ah D*(yo) + + + > (9.109) 
Solving for D?(yo), 
Diy) = ARF Me _ Fae dyy) +--+ 0.110) 
This is frequently written as 
D%(yo) = =— i gudttis - wy pos (9.111) 


where the next nonvanishing term in the Taylor series, beyond that devel- 
oped from the assumed second-order equation, is 


9, ut 
— Yoh7y, 


If, instead of adding with Eqs. (9.107) and (9.108), subtracting is 
effected, the following results: 


I 


v 
Yui — Y-1 = 2h D(yo) + a PACD, ap ee (9.112) 


eS 1 
or Wb = a a (9.113) 


Difference expressions for forward, symmetrical, and backward place- 
ment of the pivotal locations are summarized in Tables 9.26 to 9.28. 
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Figure 9.6 


Second-, third-, and fourth-order polynomials are selected for both the 
forward and the backward cases. Second-, fourth-, and sixth-order 
expressions are presumed for the symmetrical cases. Each entry con- 
tains, in parentheses, the next nonvanishing term in the Taylor series. 


9.6.2 Integration 


Equations for numerical integration are normally developed by direct 
integration of interpolating polynomials. By assuming, for illustration, 
a general Lagrange interpolation formula, and using the notation shown 


in Fig. 9.6, the equation for [i@ dz is 


fA@ dx = [ ) m2 


i=0j= one 
tj 


Site) de (9.114) 


Assuming the most elementary form of the polynomial—namely, a straight 
line passing through two points 2;, f(x) and xi41, f(vi41)—gives 


» I(x) = fe a ee a aa Slain) (9.115) 
Fist pa f(xi) % Seas ne Oo ay" 
is f(x) dz = E aoe ‘ + ae. A " 
(9.116) 
This gives 
jaarG) dx = ie aes 1) (xi — Xi41)’ a Hein! (41 a 2)? 


or [5° 1@) de ~ 50) + fea) (9.117) 
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Figure 9.7 


It is to be noted that the interval between x; and z,41 is presumed to equal 
h. This is the well-known trapezozdal rule for numerical integration. 

More accuracy could be realized by assuming that the polynomial 
passed through three equally spaced points. For the notation given in 
Fig. 9.7, the Lagrange polynomial is given by 


ee (2) aig) (2a) A G@ = 2 lea) 
nae a (fp Xo) (2-1 = 1) oa (ro — toa), aa) . 
(1 =a) (4 aa) 


TG Ge ee 


or 
x(x — h) (a + h)(x — h) (x + h)(x) 
> W@) Sa) ee Yon he Yo+ ete) Yai (9719) 
or 
Ue tee oy iaclaey, Yat — Y- 
NE a, ee ah (9.120) 


Figure 9.8 
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This gives as a numerical-integration formula 


[2 9@) de = [72 Y 0 dx =F vat 4yot vn) 0.121) 


which is known as Simpson’s one-third rule for numerical integration. By 
considering the entire range from a < x < b (see Fig. 9.8), the integral 
can be expressed as 

b h 

J) He) de = 5 ye + dys + 92) + U2 + dys td to 
se OPS. 4r ayn =F yp) | 
b h 

Jp f@) de = 5 Ya + Ayn + 2ye + dys + 2yet 
+ 2Yyn—2 + 4Yn—1 + Yo) (9.123) 


(9.122) 


It is to be noted that in this development the interval a < x < b is sub- 
divided into an ‘‘even’”’ number of segments of length h. 

The same procedure used in the development of the trapezoidal rule 
and Simpson’s one-third rule, that is, starting from the Lagrange inter- 
polation formula, can be used to develop higher-order equations. 

As noted earlier in this section, any of the interpolation formulas can 
be used to develop integration equations. For completeness and to facil- 
itate the development in a later chapter of methods of solution to differ- 
ential equations, consider here also, then, the establishment of integration 
formulas from a direct evaluation of the finite-difference formulas listed 
in Table 9.20. (It should be noted that, in general, integration formulas 
of the type and kind considered in this volume are frequently referred to 
in other books and articles as ‘“‘quadrature formulas.’’) 

Newton’s forward-difference interpolation formula (from Table 9.20) 
is defined by the following equation: 


fle + 0h) = fla) + 0as(a) + P= area 


rt = ae aa) Atf(z) + +++ (9.124) 


If it is desired to evaluate the integral of the function between the limits 
xzand x + h, that is, for the entire range between two consecutive pivotal 
points, integration is carried out with respect to @, and @ is then set equal 
to 1.0 


[PP p@ ae = [PP F@ ae, gh fy Me +) a9 (9.125) 
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or 
fla) de = hf." | f+ oaftey+ CS aye 
He et eset), 3 Mies 
= 31 A®f(x) + | dé 


6? 9 
= none) +5 afte) + Ha (20 — 30") arte 
+1 
+l, (64 — 1263 + 126%) A%f(x) + - | (9.126) 
Substituting in the limits on the integration gives the desired formula. 


[2 1@ ae = MF@) + M4 AF) + Ho AY) + 44 AY@) 
=1Voo AIC) eo ie eee 


Newton’s backward-difference interpolation formula (from Table 9.20) 
is given as Eq. (9.128). 


fle + on) = fle) + ovpe) +P EY vee) 
+ “ FOE? ven) + ++ (9.128) 


The corresponding integration formula is 
[P°F@ ae = Wyle) + M4 Wile) + 42 VI) + 36 VAC) 
A; 29)¢ 20 V2) + 25ogg Nie) | OL eg) 


For the range of integration from xp to x1, Bessel’s and Everett’s inter- 
polation formulas give the same central-difference integration formulas. 


4 _, ffoth Leth sto. 
[er 1@ ae = 1 (FS Te pie ) 


(9.130) 


It is to be noted that the first term of this series corresponds identically 
to the trapezoidal rule developed earlier in this section by using the 
Lagrange interpolation formula. 

It is frequently desirable to express integration formulas not in terms 
of differences but in terms of the function values at the fixed pivotal 
locations. For example, if only the first two terms of Eq. (9.130) are 
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considered, then by noting that 
60 = J=1 — 2fo + ys and Ot: = fo — 2fai tr S42 


the following integration formula can be obtained: 


JR $@) dx =h (2 tose iene 2) (9.131) 
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Integration formulas expressed in terms of function values at pivotal 
points, rather than as differences of these, are said to be in Lagrangian 
form. 

Integration need not be restricted to one interval. Similar equations 
can be developed for many “‘ranges of integration.’”’ Tables 9.29 and 9.30 
summarize many of the more often referred to equations of this type. 
There also are given the orders of the corresponding polynomials and the 
next terms in the series associated with each. It is to be emphasized 
that these equations assume a constant interval of h between each of the 
pivotal locations. 


STUD ¢ 


9.7 Least-squares Approximations 


In the preceding sections of this chapter it is assumed as a necessary con- 
dition that the approximating polynomials must take on the values of the 
given function at the specified pivotal points. A natural consequence of 
this presumption is that the nth-order polynomial approximating equa- 
tion requires exactly n + 1 pivotal points for evaluation. 

In dealing with experimental data or in developing other types or orders 
of approximating equations, it is often desirable to include in the con- 
sideration a larger number of points than that which would be dictated 
by direct application of the hereinbefore developed polynomial approxi- 
mations. For such cases it is desirable to specify at the outset, not the 
necessity that the polynomial pass through a given point or set of points, 
but the necessity that the sum of the errors at these points (or the sum 
of the squares of the errors between the assumed and the actual values) 
be minimized. In such a case a “‘best-fitting’’ relationship is realized. 

The method of least squares is based on the assumption that the sum 
of the squares of the deviations between the assumed functions and the 
given data, that is, the sum of the squares of the errors, must be a min- 
imum. If, as shown in Fig. 9.9, it is assumed that the ordinate of the 
data point is designated as y; and the value of the approximating func- 
tion at that same value of 2; is f(z;), the error at location x; can be written 


as 
é = yi — f(x) (9.132) 


Gennes = ie uz = up (x)f / yaxtg 9 
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Table 9.30 Nonsymmetrical (Last-interval) Integration 
Formulas (in Lagrangian Form) 


Equa- | Order of 
tion | assumed Tauation Next term 
num- poly- sd in series 
ber nomial 
Range of 
integration SS 
ae =D = O +1 
=—f_» + 8fo 7 Of41 1 
= — psf’ 
1 Second | f(z) dx =h co — oA af” (E) 
: f-2 — 5f-1 + 19fo + 9f41 19 
ah — ———— hé 
2 hird J f(z) dz =h a 750 vey 


3 Fourth i iG) ee 


S10 L0Gf-2 = 204 feu O46 fort 28M es | 20 ac 
720 1,440 


=h 


The method of least squares requires that 
» [y: — f(x)? = minimum (9.133) 
i=1 ; 


where m is the number of data points.’ 
For illustration, assume that the second-order polynomial, Eq. (9.134), 
is selected as the approximating function. 


f(x) = ax? + aye + ay (9.134) 


Figure 9.9 
© Data points - 


Approximating 
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The error at each of the ‘‘data points’ is therefore 
Error = (a2%;? + avi + ao — ys) GSH Qe em) 


The sum of the errors squared is 
S = X(error)? = S (aoa? + aya; + ao — y;)? (9.135) 
t=1 


For this to be a minimum it is necessary (but not sufficient) that 


os as 0 


passes =n) ; 
Ode 0a, 0A a) 
or yy 22;7(dex,? + AX; -+ ao — Yi) = () 
w=1 
ss 22:(aox,? — Q4X; + a = Yi) = 0 (9.137) 
i=1 


» 2(aox;? + az; + ao — ys) = 0 


i=1 


Equations (9.137) represent a set in the unknowns do, a1, and a2. Recast- 
ing these in matrix form gives 


8 
on 
& 
8 
° iS 
= 


2. 
_ 
° 
_ 
° 
_ 
° 
ry 


(9.138) 


2. 
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As a numerical example, consider the determination by the method of 
least squares of the best-fitting second-order polynomial to the data given 
in Table 9.31. The calculations are shown in tabular form. 


: Table 9.31 
x y 
0 0 
1 0.500 
2 0.866 
3 1.000 
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r ae aS a y ry xy 
0 0 0 0 0 0 0 
1 1 il 1 0.500 0.500 0.500 
2 4 8 16 0.866 1.732 3.464 
3 9 27 81 1.000 3.000 9.000 
™m 
» =6 14 36 98 2.366 5.232 12.964 


The corresponding matrix is 


98 36 14]] a, 12.964 
36 14 6]} a1} =] 5.232 (9.139) 
14 6 4) | ao 2.366 


Solution gives the values of the coefficients of the polynomial as 


a2 = —0.0915 a, = 0.6111 ay) = —0.0049 (9.140) 
The approximating polynomial is, therefore, 


f(x) = —0.09152? + 0.61112 — 0.0049 (9.141) 


Prob. 9.1: Determine any errors that may be present in the following 
table: 


| z ye: 
s fa) | Ay!) Ay | “3 
ere i cS 
2S | t. | res L = 
1.10- | 0.3043 (2S | o@ 
1.18. .| 0.3749 [0 *oe) ey = 
1.20 | 0.3899 |OTOS” fang eee 
a ofe-+ 6 Moa 
1.25 0.3044 |OOS | &a | 
1.30 | 0.4032 [O° 88, = | to 
50 ee 
1.35 0.4115 & (o 
= a y74 sar 
1.40 0.4192 0 | -4 
40 ae ae a 
1.45 0.4265 —G 
x ae OE EN re) a ——_ < 
21.50. | 0.43900 ae ae i bes hs 
oaks 
| | 
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Prob.9.2: By interpolation, determine the value of Jo(z) forx = 9.736. 


x J (x) 
0.2259 9.5 | —0.1939 
= 9.6 | —0.2090 

o(f..7 ) = 

@ 56 9.7 | —0.2218 
9.8 | —0.2323 
9.9 | —0.2403 
10.0 | —0.2459 


Prob. 9.3: For the table given in Prob. 9.2, determine x correspond- 
ing to Jo(x) = —0.2300. 


Prob. 9.4: The following tabulated data are given: 


x C2) 
—) +8.000 
ail 0.000 
0 —2.050 
+1 +1.894 


Determine by each of the following methods the value of f(x) at 
= +0.500: 
b) Gauss’ backward-interpolation formula 
(c) Everett’s interpolation formula 
(d) Method of least squares 
Prob. 9.5: Construct divided-difference tables for each of the func- 
tions A, and B,. 


nN An 15S 
2 0.416667 0.083333 
3 0.375000 0.041667 
5 0.329861 0.018750 
7 0.304225 0.011367 
10 0.280190 0.006787 
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Prob. 9.6: For the data contained in Prob. 9.5, develop Lagrange 
interpolation formulas for A, and Bn. 


Prob. 9.7: For the data contained in Prob. 9.5, develop interpolation 
polynomials by using the method of least squares. 


Prob. 9.8: Determine the best-fitting polynomial of the form 


i ax? +. Ax + ao 
for the data 


y 0.120 0.090 0.097 0.061 0.065 0.043 


Prob. 9.9: Using the approximating polynomial 
f(x) = ast + asx? + ax? + ax + ao 


and assuming a symmetrical placement of the pivotal points as shown in 
Fig. 9.5, determine the numerical-integration formula corresponding to 


iS es f(x) dx 


Prob. 9.10: Assuming forward differences and a sixth-order system, 
show that 


fp $@) de = Mohlfart Sty + fa + Ofe + fu + Sf + fe) 


Prob. 9.11: Show that 


fy He) dx = 24 5h(Tfo + 3% + 12fe + 82a + 7h) 


Prob. 9.12: Numerically integrate e~* between 0 < x < 1, in steps 
of 0.1, by each of the following: 


(a) Trapezoidal rule (b) Simpson’s one-third rule 


Prob. 9.13: To ascertain the error associated with the use of the 
trapezoidal rule for numerical integration, determine in the same manner 
as was done for numerical differentiation the next term in a Taylor series 
expansion solution. 


Prob. 9.14: For the data contained in Prob. 9.2, determine the value 
of the derivative of the function Jo(x) at x = 9.736. 
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Prob. 9.15: For the data contained in Prob. 9.5, determine the value 
of the second derivative of A, with respect to n at n = 5. 


Prob. 9.16: Given the function y = x, and the corresponding table 


x y 
—2 =) 
=] =i). 
0 0 
+1 eile) 
25 +8.0 


determine by interpolation the values of x* at 

(a) —1.60 (b) +0.05 (c) +1.50 

Prob. 9.17: By forward differences, determine the value of the second 
derivative of sin x at 


(a) x = 7/2 (bear —=10 
Assume that values of sin z are available at x = 0, 7 - = and 7. Com- 


pare the solution with exact values. 


9.8 Computer Solutions 


As pointed out in the introduction, a variety of types and methods of 
numerical analysis are considered in this chapter. Each could have asso- 
ciated with it a detailed flow diagram. It is to be noted, however, that 
many of the methods are computationally the same. Therefore, there is 
discussed in this section only the development of computer programs for 


1. Construction of a forward-difference table 
2. Development of the Lagrange interpolation formula 
3. Numerical integration using Simpson’s one-third rule 


9.8.1 Construction of a Forward-difference Table 


It is presumed that the table starts at some value x;, which has a corre- 
sponding yz, and proceeds to tn, yn. (Note: The table does not start 
necessarily atk = 0.) To facilitate computation there will be introduced 
a double subscript, for example. y,;. For the case chosen 7 denotes the 
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reference location in the table, and 7 denotes the order of the difference 
under examination. 


IS SD SSG 
0 <j < order of difference table desired 
It is to be noted that a 0 value for 7 denotes the function itself. 


The flow diagram is given in Fig. 9.10. 


Figure 9.10 
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Figure 9.11 


9.8.2 Development of the Lagrange Interpolation Formula 


This method is well suited to computer solution since it is basically a 
repetitive process. The only problem encountered is that care must be 
exercised to ensure that any given stage 7 must not equal 7. The value 
of x for which the interpolated value is desired is denoted by the letter 
u. Theinterpolated valueis Ans. The flow diagram is given in Fig. 9.11. 
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Int =Int + 2f(x) 


Figure 9.i2 


9.8.3 Numerical Integration Using Simpson’s One-third Rule 


The interval in question isa <x <b. It is assumed that the interval is 
divided into an even number of divisions n. The spacing of these divi- 
sions is denoted by h. In the case illustrated (Fig. 9.12), it is assumed 
that the function values can be computed, and need not be tabulated. 


numerical solution 
of ordinary 
differential 
equations 


10.1 Introduction 


An equation containing one or more derivatives is called a 
differential equation. If these are ordinary derivatives, the 
equation is said to be an ordinary differential equation; if these 
are partial derivatives, the equation is termed a partial dif- 
ferential equation. 

A differential equation which is of the first degree in the 
dependent variable and all its derivatives is termed a linear 
equation. Correspondingly, an equation in which the depend- 
ent variable or any of its derivatives is of the second or higher 
degree (or, more complexly, one in which the dependent 
variable is an argument of a transcendental function or the 
like) is termed a nonlinear equation. ‘The order of an equa- 
tion is the order of the highest derivative contained therein. 

One means of representing a general ordinary linear differ- 
ential equation in the dependent variable y and the inde- 
pendent variable x is shown in Eq. (10.1). 

Y, Asx) 4 + Angile) = 0 (10.1) 
i=0 dat’ 


In this case, the coefficients A,41(z) and A,(x) are shown as 
functions of x. The equation is therefore a linear ordinary 
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differential equation with variable coefficients. Any or all the A terms 
may be constants. 

An example of a second-order nonlinear variable-coefficient ordinary 
differential equation is given in Eq. (10.2). 

d? dy\3 ; 

rt + e 3) + y(sin y) = x (10.2) 
x dx 
Equation (10.3) is a second-order linear partial differential equation in 
the independent variables x and y and in the aes variable z: 


Ail) et Aste) soo + As (oy) at As (oy) = 
+ A;(z, y= es A¢(z,y) + Ax(z,y)ze = 0 (10.3) 


Here again, the coefficients have been shown as general functions of the 
independent variables, but any or all of these can be constants. When 
Ag(z,y) = 0, the equation is termed a homogeneous one. 

In most cases, the analysis of problems in engineering and applied 
science requires that the dependent variable and/or its derivatives have 
certain specified magnitudes or interrelationships at prescribed values of 
the independent variable(s). Should all these constraints be imposed at 
a zero value of the independent variable(s), or should it be possible to 
transform the given equation to one in which this is true, the problem is 
said to be an initzal-value problem. Should any of the conditions be 
imposed at other than zero, it is the boundary-value type. It is to be 
noted that problems of the mixed type can, and do, exist. One example 
is a transient heat-flow problem. In this case, spatial constraints are 
imposed at geometrical boundaries, and these may or may not be func- 
tions of time. In addition, the inztzal state, or initial values, of the prob- 
lem would be normally specified at the instant of zero time. Since the 
exact general solution of an nth-order differential equation has n con- 
stants of integration, to ensure a unique solution there must be n inde- 
pendent boundary or initial conditions associated with each problem. 

Relatively few “practical” differential equations can be solved in closed 
form. More often, solutions must be obtained by numerical procedures. 
The remainder of this and the following chapter will deal with a few of 
these methods. 

Most numerical solutions of differential equations proceed from one of 
two approaches: series expansion or determining the undefined coefficients 
that result in the least error of an assumed approximate solution. Con- 
cerning the first of these, two basically different procedures can be followed 
to obtain the series in question. First, a general Taylor series expansion 
can be written, including the specified initial conditions and the values 
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of the dependent derivatives of the function at that point. This is dis- 
cussed in Sec. 10.2. The alternative procedure for obtaining a series 
representation is to iterate successively the corresponding integral equa- 
tion, as described in Sec. 10.3. With regard to the second general 
approach, the determination of undefined coefficients based on error 
considerations, several different definitions of the significance of error 
can be considered, and four of these are discussed in Sec. 10.4. 


10.2. Taylor Series Expansion: Initial-value Problems 


The Taylor series expansion of a function in the vicinity of xo is 


(v= ta) 
PX 


NO See te eee ae yee eo) 


By denoting the interval from the original point of expansion 2x» to the 
general point x as h, Eq. (10.4) can be written in the summation form 


(h)” (=) 
h= = 10. 
Uitor B) oe nl \da) ann, (10.5) 
where 0! = 1. Given then the values of the function and its derivatives 


at a particular point, say xo, the solution for other values of x can be 
determined by suitably altering h. 


10.2.1 First-order Differential Equations 


To illustrate the general method of solution using the Taylor series 
expansion, consider the first-order differential equation (10.6), with the 
boundary condition as specified in Eq. (10.7). 

— — 0.4y3 = 1.0 (10.6) 


1.0 (10.7) 


y(1.0) 


Since the given differential equation is of the first order, only one inde- 
pendent boundary or initial condition can be specified [Kq. (10.7)]. 
Additional dependent higher-order derivatives can, however, be estab- 
lished, as is demonstrated. 

By substituting Eq. (10.7) in (10.6), the value of the first derivative of 
y with respect to ¢ at ¢ = 1.0 can be determined. It is to be noted that 
this is not an independent condition, but is a consequence of the given 
original equation and the one given boundary condition. 
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(“) = (1.0) = 1040.4 = 14 (10.8) 
dt / 1=1.0 


By successively differentiating Eq. (10.6), the required values of the 
higher-order derivatives at t = 1.0 can also be established, one at a time. 


Hh = 10+ 0.4y? a= 1.0 + 0.4(1.0)? = +1.4 
: d 
vu i=in L2y* = 1.2(1:0)7(1-4) ie = 1.68 
3 (10.9) 
wd dy\" Le 
dt? t=1 2 4 (+) oe 1 2 ag rei + 6.720 
oe) dy\* yd’y dy 
NEE ——* + 1.2y?—|,_, = +31.5840 
dt’ [t= 2.4 (2) + 7.2 dp + ae ue 


Coe atte Ae ree Oey St eedoe remy ie ke) OL cy eect oO) TOs at mete Ch OD a Oe IO Po 


To the number of derivatives calculated above, the Taylor series describ- 
ing this problem is given as 


h h? hi hi 
y = 1.0 +7 (4) + 5 (.68)-+ 57.720) + 5 (31.5840) + - - - 
(10.10) 
or y = 1.0 + 1.4h + 0.84h2 + 1.1203 + 1.3160h4 + - ~~ (10:11) 


It is to be understood that the correct solution to the problem corresponds 
to the infinite series. 

Several observations concerning Eq. (10.11) are readily apparent. 
Should h be less than 1.0, the series in all likelihood converges. Should 
a certain accuracy in y be desired, the smaller the interval of h that is 
being considered, the fewer the number of terms that are required in the 
terminated series. This is shown in Fig. 10.1, where the difference in 
using a second-order polynomial as opposed to a fourth-order equation at 
h = 0.2 is less than 1 percent. Ath = 0.1, possibly just a linear approx- 
imation would suffice. 

Another observation to be made with regard to the use of the Taylor 
series expansion is that if the differential equation is of second order, and 
if boundary conditions are specified at two separate values of the inde- 
pendent variable, the matching of this second given condition requires 
considerable additional work. 

Suppose now that it is desired to establish the function y(t) as defined 
by Eq. (10.6) in the range 1:0 < ¢ < 2.0. If itis presumed that a second- 
order polynomial is sufficient to give the accuracy needed for an interval 
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Figure 10.1 


of h = 0.2, the problem can be considered not as just one large expansion 
over the total range in question, using ¢ = 1.0 as the point of expansion, 
but rather as five successive smaller expansions: 1.0 < t < 1.2,12<t< 
Pe 1.6, 6 St <71-8, and 1:8 <7 =< 2:0. In each range the 
necessary equations are 


Ore Ge nin a 


y”’ (to) (10.12) 


where to corresponds to the starting value in that particular range in ques- 
tion. Similarly, 


y' (to) = 1.0 + 0.4 [y(to)]? (10.13) 
and y' (to) = 1.2 {Ly (to) }?y’ (eo) } (10.14) 


It is to be noted that all calculations in one subregion are needed prior to 
starting the next set. Table 10.1 summarizes the results of these calcu- 
lations for the problem in question. The corresponding curve is plotted 
in Fig. 10.2. There is also shown on the figure the exact solution of the 
differential equation. 

Table 10.1 and Fig. 10.2 suggest a still further modification of the gen- 
eral Taylor series approach as thus far discussed. At the starting point 
of the first range (at ¢ = 1.0), no information about the function is given 
for t < 1.0, and it is necessary to determine values of the higher-order 
interrelated boundary conditions in order to be able to effect the desired 
accuracy. The situation at t = 1.2 is not the same. Would it not be 
just as desirable, if not more so, to have the second range formulated so 
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Figure 10.2 


that an overlap with the first range exists? A greater degree of interplay 
between the polynomial representations of the function y in these two 
regions could then be facilitated, and the total problem [that is, y(1.0 < 
t < 2.0)] would separate to a consideration of the series representation of 
several overlapping intervals. It is to be noted, however, that the first 
region differs from the remaining ones in that special attention is required 
to obtain a sufficiently accurate starting series. Analytical continuation 
to the overlapping second, third, etc., regions is in general not as sensitive 
to round-off errors. This concept is illustrated in Fig. 10.3. 

Several different methods can be used to achieve analytical continua- 
tion. The total range of independent variable in question can, for exam- 
ple, be orderly subdivided, and polynomials developed that exactly satisfy 
the differential equation at these preselected ‘‘pivotal points.” Another 
approach is to select the undetermined coefficients of assumed generalized 
polynomials so that the total error over the entire range is a minimum. 
Still another criterion is to minimize the integral of the squares of the 
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errors. These methods for developing approximating polynomials are 
basically the same as the second general approach to the numerical solu- 
tion of differential equations, referred to in Sec. 10.1. Analytical con- 
tinuation is considered further in Sec. 10.8. 


10.2.2 Sets of First-order Differential Equations 


Sets of first-order ordinary differential equations can be solved by using a 
slightly modified form of the series method developed above. Consider, 
for example, the equations and boundary conditions defined by 


dx dy 
—S = t —= —«= — ° 
Fi 38a tyt 7 x — 2t (10.15) 


It is specified that at t = 0 
z= 1.0 and y = —1.0 


The derivatives with respect to ¢ are as follows: 


x(0) = 1.0 y(0) = —1.0 

#(0) = 32(0) + y(0) =3 -1=42.0 (0) = 2(0) = +1.0 
£(0) = 34(0) + 9) + 1 = +8.0 yO) 22 (0) 20 
= (0) = 340) + gO) = +24.0 FOV Oe sO 


Cen ein. Ce aes Cun NL Ne oN Remeber on cr cc ae bie Ghd) ie ec ete AT Ge ee 


The Taylor series expansions for x and y about t = 0 are, therefore, 


a Oise (Ot 


x(t) = 2(0) + 200+ 4 
or x(t) = 1.0 + 2¢+ 44+ 484 --- (10.16) 
and ut) = (0) + yor + ZOE EOP 

or NOUS EMI ered ate se. (10.17) 


It is to be noted that any number of equations could be solved-in this 
Same manner. As is true for the single differential equation, the error in 
the solution depends both on the number of terms carried in the series 
approximation and on the magnitude of the interval of the independent 
variable t. 
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10.3 Series Determination by Iteration of 
Integral Equations: Initial-value Problems 


For certain types of problems, where some of the successive, dependent 
derivatives become infinite, it may be impossible to obtain a Taylor series 
expansion in the manner described above. In other cases, the deriva- 
tives may be definable, but they may be too complicated or cumbersome 
to handle. In such cases, it may be more desirable to obtain an approx- 
imating series solution by successive iteration of the corresponding inte- 
gral equation. Such a procedure is normally referred to as Picard’s 
method. 


10.3.1 First-order Differential Equations 


Consider, for illustration of the general method, the first-order differential 
equation and boundary condition defined by 


y = A(a,y) —- (Xo) = Yo (10.18) 

The corresponding integral equation is 
y =yot [> Aey) ax (10.19) 

In the normal form for iteration this is 
ay = Yo bs A(x,xy) dx (10.20) 
where ;41y corresponds to the successive approximations to the solution. 


As an example, consider the differential equation and initial condition 
defined by 


y =1—xy UO) = 130 (10.21) 
The corresponding integral equation is 

y=yot fo (ay) de (10.22) 
or, in iteration form, 


Pee oes a (1 — xny) de (10.23) 


By using as a starting value 


wy = 1.0 (10.24) 
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the successive iterations yield as approximate solutions 


2 


x L 
= - = = 2 
y=10+ fa ie Meo (10.25) 
z a? a? x3 at 
y= lot fi [1-z(19+2-5)|de= 1042-4 — ee 


(10.26) 


For comparison, the Taylor series expansion is shown in Eqs. (10.27) and 
(10.28). 


y =1—xzy or y’ (0) = +1 

Oh) y’’ (0) =—1 

aye’ — —2y/’ —_— Dy y’”’ (0) = —2(1) = = 

gee = —3y" bes ay’ y’’”’ (0) — —3(—1) = 4-3 
(10.27) 


which gives 


© 
I 


ap ee ee a 
10 +7 rl) al (yee 31 Ja res)! ae 


2 3 


Hy 
0c 
eit as nes 


at 
or y Teoh te (10.28) 


It should be noted that, as expected, both procedures give the identical 
series approximation. 


10.3.2 Sets of First-order Differential Equations 


Again, to facilitate comparison of the two methods, the equations listed 
as (10.15) are solved. They are here repeated as (10.29). 


d d 
S80 oy pol <P eg = 


dt dt (10.29) 
2(0) = +1.0 y(0) = —1.0 


The general integral equations are 


20) + f) Bax tay +0 at (10.30) 


KAW 


evry = yO) + ff (ex — 28 at (10.31) 


where «41% and 441y correspond to the successive approximations to the 
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solution. These give as first approximations 


I 


10 


2 
20) + [[@-14+)d=14+ 445 (10.32) 


and ie) ibs (= 2) di = —1 +7 — 7? (10.33) 


Second approximations are obtained in the following manner: 
t 
w = 2(0) + f) (8+ 6+ 34 -—14+1—-e 4H dt 


=1+ [2+ 8+ We at 


t3 
=14 2+ 42+ 6 (10.34) 
t i 
and w= v0) + (1+ 245-2) a 


23 
eee eee z (10.35) 


Equations (10.16) and (10.34) should be compared, as should (10.17) and 
(10.34). It is to be recognized, however, that (10.34) and (10.35) 
represent only second approximations. More nearly correct solutions 
are to be obtained by continuing the iteration process. 


10.4 Polynomial Approximations 


Basically, three approaches are used in the selection of general, analytical 
approximating functions. For example, general expressions containing 
a sufficient number of unknown coefficients can be chosen which iden- 
tically satisfy the given differential equation. In this case the undeter- 
mined coefficients of these expressions are selected so that the boundary 
and/or initial conditions are best satisfied. A second approach is to 
prescribe a function that fits the boundary constraints and then, by one 
method or another, force it to satisfy (as well as can be) the differential 
equation. A third procedure is to select expressions which fit neither 
the differential equation nor the boundaries and then attempt to satisfy 
both. In the work that follows it is assumed that polynomial approxi- 
mations are sufficiently general to allow solution by any, or all, of these 
three procedures. 

In developing polynomial approximations, several points are to be 
kept in mind. It is not necessary to fit a single polynomial over the 
entire interval in question. As demonstrated in the section on Taylor 
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series expansion, piecewise approximations may be more accurate since 
each approximation is required to represent only a part of the total 
interval. Piecewise approximations are also faster to evaluate, since 
smaller-degree polynomials can be used. 

In all but the most unusual circumstances, polynomial approximations 
have errors associated with them. These deviations from the correct 
solution vary over the region in question, and the problem is to select the 
polynomial that results in the least deviation. The remainder of this 
section is concerned with various criteria that have been developed to 
minimize these errors. The four discussed are (1) collocation, (2) sub- 
division method, (3) least-squares method, and (4) Galerkin’s method. 


10.4.1 Collocation 


This method, which is extensively developed and used later in this and in 
the following chapter, requires that the error vanish at certain pre- 
selected “pivotal points.’’ For illustration of the procedure, consider 
again the differential equation and initial condition defined by (10.21). 
For convenience they are here repeated as Eqs. (10.36): 


d 
= =10—ay yOL= 10 (10.36) 


It is assumed that the range of independent variable of interest is 
Oe 0 so: 

The second-degree polynomial defined by Eq. (10.37), which contains 
two undetermined coefficients, is presumed as a sufficiently general 
approximating function. [It is to be noted that the chosen polynomial 
identically satisfies the initial condition y(0) = 1.0.] 


y=. Oar Gar? (10.37) 


Substituting from Eq. (10.37) into (10.36) results in an expression in the 
unknown coefficients a; and a2 and the independent variable z. 

The error € associated with this polynomial representation is a function 

of both the unknown coefficients a; and a, and the independent variable. 

ai(1 + x?) + a2(22 + x?) — (1 — 2x) =e (10.38) 


Assume now that it is desired to make this error vanish at the two 
interior points x = 0.25 and 0.50. For this to be possible, the two linear 


equations (10.39) must be simultaneously satisfied. 
At x= 0.25: 1.063a; + 0.516a. = 0.750 

(10.39) 
A = Ono: 1.250a, + 1,125a. = 0.500 


The solution to these equations is a; = +1.063 and a, = —0.737, and 
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the corresponding approximating polynomial is 
y = 1.04 1.0632 — 0.7372? (10.40) 


For the case just considered, because of the nature of the assumed 
quadratic equation, collocation at two points was required. The two 
arbitrarily chosen pivotal points, x = 0.25 and 0.50, were within the 
interval in question. At these points the error term e is made to vanish. 
It is to be noted, however, that while it is not forced by any algebraic 
manipulation, e at x = 0 is also equal to zero. This is a direct conse- 
quence of the chosen polynomial. 


10.4.2 Subdivision Method 


The subdivision method presupposes that the total region in question is 
subdivided into as many smaller units as there are undetermined coef- 
ficients in the assumed polynomial. For the approximation defined 
earlier as Eq. (10.37), that is, 


g=10 4 aie azz’ (10.41) 


there would be required two subdivisions. The undetermined coeffi- 
cients are selected so that the integral of the error over each of these 
smaller intervals equals zero. 


ie de = 0 (10.42) 


For the example, by assuming overlapping regions 0 < x < 0.5 and 
0.25 < x < 0.75, the two equations to be satisfied are 


0.50 0.75 
iF Peete 0 ie eae 0 (10.43) 
Substituting from the error equation (10.38) into each of these and 
integrating yields the following two linear equations in the unknown 


coefficients: 
0.542a, + 0.266a, = 0.375 


0.635a1 + 0.578a2 = 0.250 


(10.44) 


The solution is a; = 1.041 and a, = —0.711. The corresponding 
approximating polynomial is 


y = 1.0 + 1.0412 — 0.7112? (10.45) 


10.4.3 Least-squares Method 


The least-squares method requires that the integral of the square error 
over the entire range of x in question be made as small as possible. By 
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defining 
J = mean-square error = ip e? dx (10.46) 
the necessary conditions that J be a minimum are 
oJ 0 de 
— = *dz = = 10.47 
sada Ie 85g, 0. (10.47) 


An nth-degree polynomial therefore requires the evaluation of n separate 
integrations. 

Again consider for illustration of the general method of solution the 
example considered in the preceding two cases. The error is defined by 
Eq. (10.38), here repeated as (10.48). 


€ = a,(1 + x?) + an(2x + x) — (1 — 2) (10.48) 
The partial derivatives are 
0 0 
sh 1g? ee (10.49) 
Oa, 02 


The general integrals to be evaluated are 
[O° (a + 2%) + ae + 24) — (1 = 2) + 2%) dz = 0 (10.50) 


[O° (a + 2%) + axQe + 2°) — (1 — ax + 2°) dx =0 (10.51) 


The resulting linear algebraic equations in a; and a» are those given as 
(10.52). The solution is a; = 1.041 and ag = —0.715. 


1.079a; + 0.830a2. = 0.530 


(10552) 
0.830a, + 0.77la, = 0.313 
The corresponding approximating polynomial is 
y = 10+ 1.0412 — 0.7152? (10.53) 


It should be noted that in certain instances weighting functions are 
used with the least-squares method. If these functions are denoted as 
W,,, solution requires that 


J = ie We? dx = minimum (10.54) 


10.4.4 Galerkin’s Method 


Galerkin’s method, which is a special case of the more general orthogonal- 
ity method, entails that the error is orthogonal over the range in question 
to a set of linearly independent weighting functions ¢; of the independent 
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variable. That is, 
[,giedx =0 (10.55) 
Commonly in Galerkin’s method the ¢; terms are taken as the individual 


elements of the original trial polynomial. For the example problem, 
1 = x and ¢, = x”. If the original polynomial were of the form 


y = 1.0 + aye + aor? + agv? + + + > + anx” (10.56) 


n separate integrals over the entire interval would need to be evaluated. 


[pads = [pevtde = [eeide = e+ = feces =0 (10.57) 


Again, for the example under consideration [Eqs. (10.36) and (10.37)], 
the integrals are 


ei did I) Oreo yee oo TOS 


) 
fr? vlad + 2) + ae +2) — (1 = a) de = 0 (10.59) 
Evaluating these integrals gives the set of equations to be solved as 
0.3604a, + 0.3287a, = 0.1407 (10.60) 
0.1881la; + 0.1879a, = 0.0615 (10.61) 


Its solution is a; = 1.055 and a, = —0.729. The approximating poly- 
nomial is therefore 


y = 1.0 4+ 1.0552 — 0.7292? (10.62) 


Table 10.2 is a summary of the polynomial approximations that have 
been obtained in this section. The corresponding Taylor series is also 
given in the table. 


10.5 Higher-order Differential Equations: 
Initial-value Problems 


While it is possible to develop special techniques to facilitate the direct 
solution of certain types of higher-order differential equations, it is also 
possible to reduce higher-order equations to sets of first-order ones, and 
thereby allow solution by the methods hereinbefore described. Nor- 
mally, this reduction takes the form of substitution of variables in which 
the nth-order equation is replaced by a series of n first-order equations. 
The following three examples illustrate the general procedure. 
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Table 10.2 
: dy 
Given equation: re 1.0 — xy 0<2z < 0.75 
a 


Given initial condition: y(0) = 1.0 


Function evaluated at selected points. 


Method Approximating polynomial y(O) y(0.25) y(0.50) y(0.75) 
Collocation y = 1.0 + 1.0632 — 0.7372? 1.000 ~1.220 1.347 1.384 
Subdivision y = 1.0 + 1.041z — 0.7112? 1.000 1.216 1.343 1.381 
Least squares y = 1.0 + 1.0412 — 0.71522 1.000 1.215 1,342 1.379 
Galerkin y = 1.0 + 1.0552 — 0.7292? 1 O00} 213 eed 4Ommles ot 

a? 73 x4 
Taylor series 1.000 1.214 1.344 1.387 
7 x6 xi 
15h Be 105 


aie etn ee (10.63) 


Z= a 10.64) 
dx uo: 
d2Za ay 
th — = —— 
en a ere (10.65) 
and Eq. (10.63) becomes 
dz 
pe a ee (10.66) 


The second required equation to allow a solution is given by (10.64), here 
repeated as (10.67). 


sce) (10.67) 


Example 10.2 Given the set of equations 


de _ 


ey 
di y Wiad x — 2t (10.68) 
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The initial conditions are 


x(0) = 0 y(0) = -1 y(0) = 0 
If it is assumed that 


dy 
z2= — 
dt 


then, from the second of Eqs. (10.68), 


= = 2-— 2 


where Z(Ojr—= 0 y(0) = —1 2(0) #00 


Example 10.3 


d'y dy 
ORS Cee ean, 
cep ee ae 
where y(0) = Yo 
y’(0) = Yo 


af (0) = Yo. 


y"(0) = 9" 
By selecting 


_ dy 
dr 
_aA _ dy 
dx dx? 
dB d’y 


(10.69) 


(10.70) 


(10.71) 


(10.72) 


(10.73) 


(10.74) 


(10.75) 
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Eq. (10.74) reduces to 


ou oni “ say? 
es 7 (10.76) 
dB dC 
an Sa 5) 2 
dx e du y 
where y(0) = Yo 
A(0O) = ¥ 
(0) ie (10.77) 
B(O) = yo 
C0) =%_ 


10.6 Euler Methods of Forward Integration 
Many methods for the solution of the general first-order differential 
equation 
dy 
—- = fy) (10.78) 
dx 
are based on the equation 
Yin = Yi + ie f(x,y) dx (10.79) 
Often they are written in the form 
Ay = yaa — 4 = f°" fey) de (10.80) 


This section is a'consideration of the first, and most crude, of these, the 
Kuler method. It is not recommended for use except under the most 
unusual circumstances, where accuracy is of secondary importance. It is 
presented here for the purpose of indicating a general approach to the 
solution of a problem. Also, it does allow modification, as is discussed, 
and facilitates the introduction to the more usable Runge-Kutta methods 
of forward integration, which are presented in detail in Sec. 10.7. 


10.6.1 Euler’s Method 
The basic presumption of the Euler method is that the function f(x,y) can 
be assumed to be constant at its initial value f(z:,y;) over the interval of 


integration 7; <2 < 241. That is, 


Ay = Yit1 — Yi = Yorn — Ys = ky (10.81) 
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where ky = Alf(aiy:)] (10.82) 


This can equally well be written 
Ay = y+ h) — ya) = hy (10.83) 
1 = Alf(z,y)] (10.84) 


The results of this type of reasoning can be seen from Fig. 10.4. If 
y’(x) > 0, then the Euler method yields a value for y(z + h) which is less 
than the correct value. For y’’(x) < 0, the value obtained is too great. 

In terms of overall general accuracy of solution, an improvement in the 
above-defined method is realized if the function is considered at some 
intermediate point x; < x < 2:41; for example, at the midpoint of the 
interval. The governing equations are 


Ay =i (10.85) 
and ky = h{flx + Yh, ya + Wh)]} (10.86) 


The major difficulty associated with this improvement is emphasized by 
the brackets and parentheses contained in Eq. (10.86). y, which is needed 
to establish the function value f, is a function of the independent variable 
x. Therefore, f is not readily defined. 


10.6.2 Modified Euler-Cauchy Method 


Another method for improving the Euler method can be realized by 
averaging the slopes of the function y(x) at its end points, and by pre- 
suming that this average value is constant over the entire interval h. 
Unfortunately, the same problem exists as discussed in the preceding case. 
The function value, and therefore the slope of the function at x + A, is not 
readily available. Approximations to it can, however, be established. 


Figure 10.4 


Given function y(x) 
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In the modified Euler-Cauchy method it is assumed that 


ay = 5 lesu) + Fenn] (10.87) 


where f(2i+1,Yi41) is an approximation to the true value of the function at 
a +h. It is further presumed that the Euler method [Eqs. (10.83) and 
(10.84)] is sufficiently accurate to establish this approximate function 
value. In the notation of the preceding case, the modified Euler-Cauchy 
method is defined by Eqs. (10.88) and (10.89). 


Ay = (ki + ke) (10.88) 
ky = A{f(z,y)] 


As an example of the numerical procedure that results in the solution 
of problems by use of this method, consider the evaluation of (0.5), given 
the differential equation 


(10.89) 


y’ = f(x,y) = 1 — ay (10.90) 


It is specified that the initial condition is y(0) = 1.0. For a “‘one-step”’ 
forward integration the interval h would be selected as the full value of 
the independent variable. Thatis,h = 0.5. From the first of equations 
(10.89), 


f(x,y) = f(0,1.0) = 1 — (0)(1.0) = 1.0 


Therefore 


ky = Alf(z,y)] (0.5) (1.0) = 0.5 (10.91) 
The second of Eqs. (10.89) is evaluated in the following sequence: 


zth=0+ 0.5 = 0.5 


y+tkh =104+ 0.5 = 1.5 
and fe + kh, y - hay = (051.5) = 1 — ay 
= 1 — (0.5)(1.5) 
= 0.25 
Therefore 
ke = Af@ + hy y + I) 
= (0.5)[f(0.5,1.5)] = (0.5) (0.25) = 0.125 (10.92) 


From Eq. (10.88) 


Ay = 4(ki + ke) = 14(0.5 + 0.125) = 0.3125 (10.93) 
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V(x+h) 


Figure 10.5 


This gives as a value of the function (0.5) 
y(0.5) = y(0) + Ay = 1.0 + 0.3125 = 1.3125 (10.94) 


It is evident from the above that the methods presented in this section 
result in an accuracy of solution which depends primarily on the selected 
interval h. Accuracy, however, also depends on the overall behavior of 
the function in question, as is now demonstrated. Consider the three 
cases shown in Fig. 10.5. In each it is presumed that the function starts 
at x, y(x) and ends atx +h, y(x +h). In case A it is assumed that in 
the interval in question y’’(z) > 0. For case B, the presumption is that 
y’'(x) <0. Case C presupposes a point of inflection within the interval. 

Figure 10.6 illustrates graphically the procedure that is followed in 
obtaining an approximate value for y(x + h) by using the Euler-Cauchy 
method. The initial values x and y are presumed given. On the figure 
they are shown as location @). With these starting values and the given 
first-order differential equation, it is possible to establish the slope of the 
function at @. That is, y’ = f(z,y). A first approximation to the value 
of y at x + h can be realized by assuming, as in the Euler method, that 
y’ remains constant at its initial value, over the interval h. (This is 


Figure 10.6 
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shown as location @ on the figure.) With this first approximation to 
the value of y(z + h), an approximate slope at x + A can now be deter- 
mined, f(z + h, y +1). A second approximation to the true solution 
y(z + h) can be obtained by averaging the slopes at @ and @) (that is, 
weighing them equally) and using this average value over the entire inter- 
val. Location @) would be the resulting predicted solution. It should 
be evident that for case A type curves [y’’(x) > 0], location @) is always 
a closer approximation to the true solution than is location (). 

The Euler-Cauchy method applied to case B type curves is illustrated 
in Fig. 10.7. Again, location @) is a better approximation than is loca- 
tion (2). 

For case C, where y’’(x) = 0 at some point within the interval in ques- 
tion, it is not possible to reason that location @) is always a better approx- 
imation to the true solution than is location @). Moreover, if the slopes 
at @ and @ are of opposite signs, @) may be appreciably better than @). 

There are several ways to improve the accuracy of solution of the meth- 
ods developed in this section. For example, the interval h could be sub- 
divided into several smaller regions and there could be developed algebraic 
equations which, in turn, apply the Euler-Cauchy method to each sub- 
division and sum the resulting values. However, very little, if any, com- 
putational advantage would be realized by such a procedure over just 
selecting a smaller interval h and repeating the process the desired number 
of times. Such a method is not developed here. 

From Figs. 10.6 to 10.8 it is evident that the slopes of the functions at 
the terminal points of the interval in question may not be necessarily the 
best indicator of the “effective average slope’ over the interval itself. A 
second method for improving the accuracy is therefore suggested: a 
weighted averaging of approximate slopes at several different locations. 
Assume for illustration of the general method that the predicted change 
in y over the interval A (that is, Ay) can be defined in terms of approxi- 
mate (or exact) slopes at four different locations. 


Figure 10.7 
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L ees ge Case C 
x x+h 
x 
Figure 10.8 
Ay = ak, + bk + ck + dks (10.95) 


where a, b, c, and d are weighting values and ky, ko, k3, and ky are h times 
the slopes at four points within the interval in question. If it is assumed 
that all the slopes are to contribute in a positive fashion, the weighting 
functions must be 1, or less than 1, and must sum to 1.0. That is, 


at+b+ct+d=l1 (10.96) 
and OS NAY) b=— 11..0 @ =< io aS il (10.97) 


The computed approximate slopes used in determining k; through k, could 
be defined in one of several ways. For example, each could be prescribed 
using the Euler method, assuming as a starting point the initial condition 
Pale 

ky = h{f(a,y)] 


ks = h[f(x + ph, y + pka)] 


However, if it is specified that m < n < p < 1.0, possibly a more realistic 
solution would be realized by using the following definition of terms: 


ki = Alf(a,y)] 

ky = Alf(x + mh, y + mki)] 
ks = h[f(w + nh, y + nkz)] 
kg = A[f(a + ph, y + pks)] 


Other definitions could equally be specified, the accuracy of the solution 
depending not only on these equations but also on the selected weighting 
functions a through d. 


(10.98) 


(10.99) 
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10.7 Runge-Kutta Methods 


10.7.1 First-order Equations 


Higher-order approximations of the basic forward-integration type dis- 
cussed in Sec. 10.6 can be developed in a more general form by using the 
methods of Runge-Kutta. These methods provide solutions which are 
comparable in accuracy to Taylor series solutions in which are retained 
many higher-order derivatives. They are much used for accurate 
integration. 

Again, it is presumed that the differential equation in question is of the 
first order and can be expressed in the following form: 


WL ee (10.100) 
dx 


It should be noted that this equation need not be linear. A Taylor series 
expansion about point x, y of the function y(z) is 


y(a +h) = ye) + hy’) ie "(a) agi On) = eer UA 101} 


2! i¥ 3! iv 
where A is the interval of expansion in question. This equation can 
equally well be written in terms of the change in y, or Ay, associated with 
a change in x from x to x + A. 


Ay = y(x +h) — y(x) = hy'(x) + 5y"(@) a) O02} 


si¥ 


It is to be recalled that the Taylor series method for obtaining starting 
solutions allows the definition of higher-order dependent derivatives. For 
the given differential equation (10.100), the second derivative of func- 
tion y(x) with respect to z is 


of(z,y) 4 of(x,y) dy 


Die Ox dy dx 


(10.103) 


But from the differential equation, dy/dx = f(x,y). Therefore, Eq. 
(10.103) can be written as 


OW “ew 4 f(a) ee y) flay) (10.104) 
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For compactness, the following notation is used in the development of 
the Runge-Kutta methods: 


f(x,y) =f 
of (x,y) tes Ofz a2 Of rx 
ax = ip ax 3 ie ae =e Vase 
of(x,y) 7 ofy a Ofyy a 
ay = ty ay es Suv Oy < Suvy (10.105) 
die ee 
ay ae Sey oy — Sev 
The derivatives can then be expressed as follows: 
y =f (10.106) 
Y= fo + Suk (10.107) 
YY” = fez + 2fovf + ff? + fulfe + ff) (10.108) 


ae - Sean 3 Sfrryf st3 Sfews? te Sunf? =f Fufes =e faut a5 Su’) 
+ 3(f2 + AS) feu + fut) + fn (Fe + ff) (10.109) 


In turn, Eq. (10.102), the Taylor series, will be 
h? h3 
Ay = hf oh 9 (fz + ff) Ze 6 [fra = 2fauf ot Suef? + ful Se + frf)] 


h4 
ig [fear + Bfzevf + 3fewS? + fowl? + fufee + 2feul + Swf?) 


+ 3(fr + fuf) (few + furf) + Suv? (fe +S)! aba (10.110) 


The Runge-Kutta methods provide a straightforward means for devel- 
oping expressions for Ay in terms of values of the function f(x,y) at various 
locations along the interval in question. Moreover, the resulting formu- 
las satisfy Eq. (10.110), the Taylor series, to the accuracy desired. 

As in Eq. (10.95), it is assumed that Ay can be expressed in the follow- 
ing form: 

Ay = ak, + bk + ck + dks (10.111) 
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where, by definition, as in Eq. (10.99), 
ky = hf(z,y)] 
ke = h[f(x + mh, y + mk,)] (10.112) 
ks = Alf(a + nh, y + nk,)] 
ka = hif(x + ph, y + pks)] 


Each of these equations (10.112) can be expanded in a Taylor series in 
two independent variables. That is, 


ks = Hie +a, y + 8) (10.113) 
where 
fle +a, y + B) = fey) + ofs(a,y) + Bhi(@y) 

+ 5 la%clty) + 2oBfev(y) + BYfw(ew)l + «++ (10.114) 


By using the notation of Eqs. (10.105), this is written as 


HH 
Tl Oy eee) = f+ of. + Bf, + 51 (fee + 2aBfey + Bfw) A ee is 


(10.115) 

The corresponding expression for kz is 
a = mh (10.116) 
B = mk, = mhf (10.117) 


which gives 


ke = hif + mhfz + mkify + 4 (mP*h*fer + 2mhmkifey + mk fy) 


+ ee} (011s) 
or 


2f2 
kp =h Ei a mh(fr + ff) = > ce + 2fayt ats Swf?) 
ots a (frsx + 3fseyf + Sfruyf? + Sun¥?) i | (10.119) 


Expressions for ks and k, are to be obtained in a similar manner. The 
resulting expressions are listed in Table 10.3. 

It is the primary purpose of the Runge-Kutta methods to establish the 
interrelationships between the constants a, b, c, d, m, n, and p such that 
Eqs. (10.110) and’(10.111) correspond to the degree of accuracy desired. 
This could be done in a number of ways. The method selected is to have 
the coefficients of h, h?, h3, and h4 in both systems equal. For example, 
the coefficient of h in Eq. (10.110) is f. The term in the expanded series 
corresponding to Eq. (10.111) is af + bf + cf+df. It is therefore 
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required that 
PGi UL acy ot Gy 


10. 
or a+t6ob+c+d=1.0 oy 


In like manner, matching the coefficients of A? in each system, 


rae + ful) = bm(fz + fuf) + en(fe + ff) + dp(fe + fuf) (10.121) 
or bm + cen+dp=14 (10.122) 


Table 10.3. Summary of Coefficients 


h?2 h3 ht 


0 Ot 0 


m?* m3 
ke = f | m(fz + fuf) oF Gaz + fest + furf?) 7 (frre + Sfeayf 
+ 3feyyf? + fof?) 
k3 = f \n(fe + fuf) = (fez + 2fzvf + fuvf?) fess + 3frzyf + 3feyryf? 
+ mnfy(fz + frf) + funyf®) + 19m 'nfy (fez 
+ YFaf + ful) 
+ mn? (fey + fot) fe + ff) 
ky = f | pte + fuf) = (frr + 2fzvf + fuvf?) (fee + 3frzuf + 3feyyf? 


+ npfy (fz + fuf) + fuvyf*) + len? pfy (fez 

+ 2fryf + fuvf?) + np? (fey 
+ fuvf) (fe + fof) 

+ mnpf,?(fe + ff) 


For the coefficients of h’, it is presupposed that the necessary conditions 
can be effected in two separate steps, as is now demonstrated. The total 
equation is 


Ye (fez + 2fayf + Swf’) tg Ve fut + fuf) a V6 (fez + 2faf + favf?) 
xX (mb + nc + pd) + [fulfz + fuf)Momn + dnp) (10.123) 


This equation, however, can be satisfied by subdividing it into the follow- 
ing separate equations: 


V6 (fee + 2feyf + Sif?) = (feo + 2fevf + Fiad )\nb) ne + pd) 
(10.124) 
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and Voli(te + A) = (te + ff) omn + dnp) (10.125) 
These give Eqs. (10.126) and (10.127). 

mb+ne+ p'd='4 (10.126) 

cmn + dnp = (10.127) 


Table 10.4 summarizes the “sufficient”’ interrelationship, developed in 
the manner just described, that must exist to effect a solution having an 
accuracy up to the order of h®. Any set of values satisfying these makes 
it possible to obtain a solution. 


Table 10.4 Sufficient Conditions 
for Runge-Kutta Methods 


Order of | Equations for determining 


accuracy coefficients 
h a-b-+e¢-—-d= 1 
h? bm + cn + dp = 16 
h bm? + cn? + dp? = 14 


cmn + dnp = 16 


h4 bm? + cn? + dp? = 4 
cmn? + dnp? = 1¢ 
cm’n + dn?p = 149 
dmnp = }24 


As an example of how the information contained in Table 10.4 might 
be used to develop a set of Runge-Kutta type equations, assume that it 
is desired to develop an expression involving only two terms, which has 
an accuracy of the order of h®. That is, the desired equation should be 
of the form 


Ay = ak, + bko (10.128) 
with ky ania.) | kp = h{f(a + mh, y + mk)] (10.129) 
Using the equations hereinbefore developed, it is required that 
c=d=n= 7 =0 (10.130) 
The required expressions, from Table 10.4, are 
a+b6=1] (10.131) 


bm = 14 (10.132) 
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Selecting m = 1 yields values for a and b as 


/ 


m=%7Fa=% b=% (10.133) 


It is evident that such a selection corresponds to the Buler-Cauchy method 
defined in Sec. 10.6.2. [It is to be recognized also that this is not a unique 
set for the three-variable two-equation system of (10.131) and (10.132).| 

A Runge-Kutta set of equations which is accurate to the order h*, and 
which presumes the following values of the constants, 


a= m=34 

b=0 n= 24 

ey en (10.134) 
d= 0 


satisfies the “‘sufficient’’ equations listed in Table 10.4. The resulting set 
of expressions, known as Heun’s method, is 


Ay = }4(ki + 3ks) 

ky = h[f(z,y)] 

ke = h[f(x + Yh, y + 16h)] 
ks = h{f(x + 24h, y + 24ke)] 


(10.135) 


Several of the more commonly used Runge-Kutta methods are given in 
Table 10.5. 

To give an indication of the accuracy of these various methods, and to 
compare them with the corresponding Taylor series expansion, consider 
the numerical solutions of the differential equation 


y= ry (10.136) 


As before, it is specified for all cases that y(0) = 1.0. The results are 
given in Table 10.6. 

It is pointed out above in this section that, like most other numerical 
procedures for solving differential equations, the methods of Runge-Kutta 
are sensitive to variations in the selected interval h. Too small a value 
leads to an excessive number of computations, with many digits retained. 
On the other hand, too large an interval may well lead to an answer that 
differs significantly from the true one. 

A method often used to examine the accuracy of a solution is to first 
solve the given problem for a trial selected interval A and then repeat the 
process, through two cycles, for an interval 19h. It should be noted that 
such a procedure has the added advantage of providing a “current” check 
on the numerical calculations. As an example of the influence of “halv- 
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Table 10.5 Runge-Kutta-type Equations for y' = f(x,y), 
y(x + h) = y(x) + Ay 


Method Equations 

Euler INI ssi 

(Error of the order of h?) ky = hlf(z,y)] 

Euler-Cauchy Ay = (ki + ko) 

(Error of the order of h’) k, = Alf(z,y)] 
li aCe Se 1, Wf a= le) 

Heun Ay = 44(ki + 3ks) 

(Error of the order of h4) ki = Alf(z,y)] 


ky = h[ f(a Se YAO, Wl se 14k;)] 
ks = h{f(x.+ 2h, y + 3k2)] 


Kutta-Simpson Ay = L6(ki + 2k2 + 2k3 + ka) 
(One-third rule) ki = A{f(z,y)] 
(Error of the order of h5) ko = Alf(x + Wh, y + 16h) 


ks = hif(@@ + Wh, y + 14k) 
kg =hf@ +h,y + ka)] 


Kutta-Simpson Ay = 1é(ki + 3k2 + 3k3 + ka) 
(Three-eighths rule) ky = hif(z,y)] 
(Error of the order of h*) ke = A[f(x + 3h, y + 3h1)] 


ks = hl[f( + 26h, y + ke — 13h1)] 
ka = Alf +h, y + ks — ke + ky)] 


ing the interval,’”’ again consider the first-order differential equation 
Gosia ee (10.137) 
with y(0) = +1.0 (10.138) 


The Kutta-Simpson one-third rule equations are used for illustration. If 
h is selected equal to 0.25 and if expansion about x = 0 is presumed, the 
necessary terms in the solution are 


ky = 0.25[f(0,1.0)] = 0.25 

ke = 0.25[f(0.125,1.125)] = 0.214844 

ks = 0.25[f(0.125,1.107285)] = 0.215393 

ks = 0.25[f(0.25,1.215397)] = 0.174038 

Ay = 16(0.25 + 0.429688 + 0.430786 + 0.174038) = 0.214085 


and y(0.25) = y(0) + Ay = 1.00 + 0.214085 = 1.214085 
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Table 10.6 


nn 


Function evaluated for 
selected intervals 
of expansion 


Method General equations y(O) y(0.25) y(0.50) ¥(0.75) 
——————— 
Euler Ay = ki UDO Th PO) LOO) 17 
EKuler-Cauchy Ay = 16 (ki + ke) TOCO M22 Ries Tome 258 
Heun _ Ay = 34(k: + 3ks) 1,000 41,2127 1.342 1.372 
Kutta-Simpson Ay = 16(ki + 2ke + 2k3 + hs) 1.000 1.214 1.343 1.390 


(One-third rule) 


Kutta-Simpson Ay = 16 (ki + 38ko + 3ks +s) 1.000 1.214 1.343 1.385 
(Three-eighths 
rule) 

F Caan ee = 
Taylor series Ay =z —— —— +— 1.000 1.214 1.344 1.387 
solution 2 3 8 

ae ak Be 
hig 13105 = 


Table 10.7 summarizes this and other solutions. Note the comparable 
function values, using both a one-step application of the Runge-Kutta 
equations and a two-step application. 


Table 10.7 dy/dx = 1 — zy, yO) = +1.0 


Selected interval Function values 


of expansion 


h y (0.25) y (0.50) y (1.00) y (2.00) 
0.25 1.213991 1.342720 
0.50 1.34395 1.31824 
1.00 1.3229 +0.7474 
2.00 —0.3333 


a 


10.7.2 Sets of First-order Equations 


Forward integration methods of the Runge-Kutta type can be developed 
to facilitate solution of sets of first-order differential equations. For two 
equations the relationships summarized in Table 10.8 are often used. By 
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comparing these with that developed for single equations (see Table 10.5), 
it is evident that the method can be extended to larger numbers otf 
equations. 


Table 10.8 
, ee eee 
Given differential equation Given differential equation 
Ge) Y= g(t,2,y) 
Aas pay A JU, ZY 
ky = Alf(t,z,y)] f, = h(g,z,y)] 
ke = Alftt + 16h, 2 + 16k1, Y+360)) | & = Alot + 4h, & + Mk y + 124)] 
ks = Alf(t + 16h, t + ko, y +1 O)) | ts = Alot + Mh, & + Moko, y + 344)] 
ka = hUf(t th, e + ks, y + 43)] t, =hig(t + hy & + ks, y ts) 
Ac = 16 (ky + 2k» + 2k3 + ka) Ay = Ve (4 == 2f5 otal 2; se £4) 
a(t +h) = x(t) + Az yi +h) = y@ + Ay 


Consider, for illustration, the solution of the two first-order equations 
shown as (10.139). [These are the same equations that were solved ear- 
lier by using the Taylor series. See Eqs. (10.15).] 


Fieger ar 
(10.139) 
ce eS ee 
a 
where x(0) = +1.0 and y(0O) = —1.0 


If h = 0.2 is selected, solutions are as follows: 


k; = 0.2[f(0,1.0,—1.0)] = +0.400 
f, = 0.2[g(0,1.0, —1.0)] = +0.200 
ke = 0.2[f(0.1,1.2,—0.9)] = +0.560 
fo = 0.2[9(0.1,1.2, —0.9)] = +0.200 


ks = 0.2[f(0.1,1.28, —0.9)] = 0.608 
£,; = 0.2[9(0.1,1.28, —0.9)] = +0.216 


ks = 0.2[f(0.2,1.608, —0.784)] = +0.846 
£, = 0.2[9(0.2,1.608, —0.784)] = +0.242 


Ap= 6 i GG 20 ten One 
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or 


(0.2) = +1.0 + 0.596 = 1.596 
y(0.2) = —1.0 + 0.212 = —0.788 


The comparable Taylor series (expanded about ¢ = 0) for the given 
differential equations and initial conditions are 


x(t) = +1.0 + 2 + 4t? + 448 + 104¢4 + 33/545 + 1096578 4 - - - 
(10.140) 


y(t) 
eeu — OY» 


—1.0 +44 4408 + t4 + 2409 + 83g qt% + + - (10.141) 


x(0.2) = +1.5981145 y(0.2) = —0.7874968 (10.142) 


Table 10.9 summarizes the solutions for t = 0.2, 0.4, and 0.6. It is to 
be noted that all the values shown for the Taylor series are obtained by 
x, presuming t = 0 as the point of expansion. This was not the case for 
the Runge-Kutta solutions. There h was selected equal to 0.2, and the 
reference location was different for the three cases tabulated: 


y(0.2) = y(0 + h = 0.2), y(0.4) = y(0.2 +h = 0.4) 


and y(0.6) = y(0.4 + h = 0.6) 
SiG =a rene ay 
* = SS 40; 
cca dt 4 2(0) = +1.0 
dy y(0) = —1.0 
eat 
dt is 
Function Taylor Runge- 
values seriesf Kuttat 
(0.2) +1.5981145 +1.5956 
y (0.2) —0.7874968 —0.7877 
x(0.4) +2 .8088216 +2.77361 
y (0.4) —0.4807381 —(0.45857 
x(0.6) +5.16358 +5 .00435 
y (0.6) +0 .0865472 +0 .06974 


Ce ————————es 
} All Taylor series values are obtained by assuming 
expansion about ¢ = 0. 

+ All Runge-Kutta solutions are obtained by assum- 
ing h = 0.2. 
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10.7.3. Higher-order Equations 


It is demonstrated in Sec. 10.5 that higher-order differential equations 
can be treated as if they were sets of first-order equations. Runge-Kutta 
type forward integration solutions can be obtained, therefore, in the man- 
ner just described. It is evident, however, that such procedures are not 
the only ones available. More direct solutions can be obtained by repeat- 
ing the whole process hereinbefore described for first-order cases. 

The necessary Runge-Kutta equations for solution of a general second- 
order equation are summarized in Table 10.10. These are obtained by 
direct development, proceeding in the fashion described for the first-order 
case in Sec. 10.7.1. 


Table 10.10 y' = f(z,y,y’) 
h2 
ky = 9 [f(ayy)) 
h?2 1 he 1 ; 1 
ke Es s(2 a y tT 35Y Sp B) +m) | 
h? 1 | teres 8 eee 
ks -Eli(2 Tah, y +5y “arp y +im)| 


h? 2 
n= El s(z thy +hy’ + ks, v+im)| 


il 
Ay = 4 (hi + he + ks) 


1 
Ay’ oa 3h (ky + 2ke + 2k3 + ka) 


y(x +h) = y(x) + Aly’(x)] + Ay 
y'(c +h) = y'(z) + dy’ 


To illustrate the procedure of.solution using the equations contained in 
Table 10.10, consider the solution of Eq. (10.143), 


y” = 3xyy’ — (1+ 2)y (10.143) 
where y(0) = 0 and y'(0) = +1.0 (10.144) 


For the case where h = 0.2, the Runge-Kutta solution is obtained as 
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follows: 


ky = 92" (700,0,1.0)) = 0 


ky = 0.02[f(0.1,0.1,1.0)] = —0.0016 
ks = 0.02[f(0.1,0.1,0.992)] = —0.0016048 
ks = 0.02[f(0.2,0.198,0.984)] = —0.0024142 


1 
Ay 3 (0 — 0.0016 — 0.0016048) = —0.0010683 


1 
Ay’ = 06 (0 — 0.0032 — 0.0032096 — 0.0024142) = —0.014706 


y(0.2) = y(O) + 0.2[y’(0)] + Ay 
= 0 + 0.2(+1.0) — 0.0010683 = +0.198932 (10.145) 
y' (0.2) = y’(0) + Ay’ 
+1.0 — 0.014706 = +0.985294 (10.146) 


For comparison, a Taylor series solution of Eq. (10.143) proceeds in 
the following fashion. Let 


g2=y7 (10.147) 
Then 2’ = 3ayy’ — (14+ a)y (10.148) 


Independent and dependent boundary conditions are developed in Table 
TOSI. 


Table 10.11 
y'(z) =2 z'(x) = 3zyy’ — (1 + a)y 

y(0) = 0 [boundary condition] z(0) = +1.0 [boundary condition] 
y'(0) = 2(0) = +1.0 2'(0) = 8ayy’ — (1 + a2)y =0 
y"(0) = 2’/(0) = 0 2'(0) = —1.0 

y’"(0) = 20) = —1.0 2'"(0) = +4.0 

yy" (0) = 2’"(0) = +4.0 2”"(0) = —1.0 

Since 
y"'(0) ~ y’"’ (0) AY) 


POOP eae toe eg et 
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and 
2’’(0) ZO) PEGA AD or A 
z= 2(0) + 2’(O)z + 51 bade 31 ba ans = 
the series representations are 
Yo APO VEE Gl Bae (20) ee (10.149) 
and yf = 2) = <1 0 on? a ear (10.150) 


This gives for comparison with Eqs. (10.145) and (10.146) the following 
series-determined values at x = 0.2: 


y(0.2) = +0.19893 (10.151) 
and y'(0.2) = +0.98527 (10.152) 


The comparisons for these and other values of h are shown in Table 10.12. 


Table 10.12 
a8 Taylor series solution | Solution using Runge- 
function | y and y’ fune-| using Eqs. (10.149) Kutta equations, 
values | tion values and (10.150) Table 10.10 
eS HOD) = +0.19893 +0.19893 
Y(0.2)e— +0.98527 +0.98529 
0.4 y(0.4) = +0 .39352 +0.39352 
5 . y'(0.4) = +0.96163 +0.96346 
git y(0.6) = +0.58496 +0.58502 
; y' (0.6) = +0.96860 +0.95351 


10.8 Analytical Continuation 


It is evident from the preceding examples in this chapter that accuracy 
of solution can be obtained by the methods heretofore developed only 
if the range of independent variable under study is kept small. This is 
due primarily to the fact that as the size of the increment is increased, 
more terms must be-included to maintain the same degree of accuracy. 
This almost always involves a vastly increased amount of computation. 
To circumvent this situation, initial-value problems are normally solved 
as two-stage problems: first, starting the solution, and second, analytical 
continuation. 


The starting procedure has as its primary purpose the establishment 
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of a few accurate values of y = f(x) at preselected pivotal values of x. 
Since when z is held to a small interval, “accuracy” can be obtained with 
relatively few terms included, almost any of the methods hereinbefore 
described can be used for starting the solution. 

The method selected for continuing the solution must be such that the 
accuracy of the first few values (obtained as described in the preceding 
paragraph) is maintained. Moreover, the method should not be too awk- 
ward to use, and must allow solution over an extended range of x. Pro- 
cedures of this general type are illustrated in Fig. 10.3. They are fre- 
quently referred to as marching methods, since each step overlaps the 
others in such a way that only one value of y is unknown at a given time. 

In general, methods for analytical continuation are of the finite-differ- 
ence type; that is, they involve the use of function values (or their cor- 
responding differences) at pivotal points. Two distinct and separate pro- 
cedures have been extensively used: (1) predictor-corrector methods and 
(2) collocation methods. 


10.8.1 Predictor-Corrector Methods 


The basic concept of a predictor-corrector solution requires a two-stage 
operation. The first stage is an approximate determination of the solu- 
tion using a so-called ‘‘open integration formula,” that is, an integration 
formula that involves known values ‘“‘to the left” of the point being sought. 
The second stage makes use of the result of the first calculations in a 
second, more exact, ‘‘closed integration formula.” 

To illustrate the basic concepts involved, consider the first-order dif- 
ferential equation 


d 

“f= f(x,y) (10.153) 
dx 

It is assumed that the range of solution of interest is from x) to x» + h = 21. 

It is also presumed that the Euler and Euler-Cauchy equations from Table 

10.5 are available. The first of these, 


y(a1) = y(to) + Ay = y(o) + Af(%o,Yo)] (10.154) 
is of the “open” type. (Note that all values on the right-hand side of 


the equation are concerned with location x, not x1.) The Euler-Cauchy 
equation written in the same form is 


y(t1) = y(to) + 44 {Alf (@o,yo)] + Af(e1,y(er))]} (10.155) 


It is to be observed that in this case both sides of Eq. (10.155) involve 
y(x1). tis therefore said to be of the “closed” type. Referring to Table 
10.5 shows that the error associated with Eq. (10.154) is of the order of 
h2, whereas that for Eq. (10.155) is of the order of h*. 
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It is possible to define a well-ordered iterative-solution procedure by 
using Eqs. (10.154) and (10.155). If the predictor value is denoted with 
an asterisk, ‘‘*’’, the two equations are: 


Predictor: y(x1)* = y(to) + Alf(xo,yo)] (10.156) 
y (to) + Lehlf(xo,yo) + f(r1,y(a1)*)] (10.157) 


First, y(z1)* would be predicted from (10.156). This value would then 
be substituted into Eq. (10.157) and a new y(x:) would be computed. 
Should the predicted and computed values differ too greatly, Eq. (10.157) 
would again be used, but this time with the y(x1) just computed as a new 
predicted value. This iteration process would be continued until the 
desired accuracy was achieved. 

The best known of the predictor-corrector methods are those due to 
Milne. In general, they are based on integration formulas of the type 
considered in Chap. 9, which are summarized in Tables 9.29 and 9.30. 

By considering only those integration formulas in Table 9.29 whose 
errors are of the order h®f’’’’(é), the following can be listed: 


Corrector: y (x1) 


OprpEN EQUATIONS 
From Eq. (10), Table 9.29: 


7d oats a 2100 


Yur =Yo9 Pah 3 


zh f'(2) ——-- 0.158) 


From Eq. (13), Table 9.29: 


a py eon eee ee 


= asp" (E) (10.159) 


24 
CLOSED EQUATIONS 
From Eq. (4), Table 9.29: 
1 + 4fo + 1 
Yui = Y-1 + ont : ® Fa " hef’’""(£) (10.160) 


From Eq. (8), Table 9.29: 


a 3) ae 3 
var = yaa + 3h EAE Ot fo asper(e) (10.161) 


Comparing the error terms shows that Eq. (10.158) is clearly superior to 
Eq. (10.159) as a predictor equation. Similarly, Eq. (10.160) should be 
selected as the corrector formula. [Note that the error in the selected 
corrector equation (10.160) is less than that in the predictor equation 
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(10.158).] Therefore, 


2f-2 — f- 2 

Yai = y-3 + 4h Ja 7 + fo (Predictor) (10.162) 
By 4 

UES 6 PES ae ane is + fia (Corrector) (10.163) 


Predictor-corrector methods and formulas are also defined in terms of 
differences. ‘The Adams-Bashford method, for example, expresses the 
equations in terms of backward differences. 


Yo + h(fo + KVfo + 4oV7fo + 36V%fo + 2514 o0V4fo + 954g8V*fo 
a) ei) (Predictor) (10.164) 


Yee Yo hi — 14Vir— V7 — afi — 2400Vi — AeoV'i 
2st) (Corrector) (10.165) 


Y41 


As an illustration of the use of Milne’s method, consider the continua- 
tion of the solution to 
d 
Nee ay) (10.166) 
dx 
where the three initial points were obtained by Heun’s method and are 
given in Table 10.13. Only the definition of y at x = 0.8 is illustrated. 


Table 10.13 
dy 
c y f(z,y) = a 
ke 
0 1.0000 1.0000 < (Given boundary condition) 
0.2 1.1775 Pais (Obtained by using 
0.4 1.3020 0.4792 Houn’a method) 
0.6 1.3672 0.1797 


.2,1.1775) — f(0.4,1.3020) + 2f(0.6,1.3672 
HE Oe asks : ) + 2f( ) 


1.0000 + ; (0.2)[2(0.7654) — (0.4792) + 2(0.1797)] 


1.0000 + 0.3758 = 1.3758 (Predictor) (10.167) 
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Substituting this value into the corrector equation, 


f(0.4,1.3020) + 4f(0.6,1.3672) + f(0.8,1.3758) 
3 


= 1.3020 + 14(0.2)[(0.4792) + 4(0.1797) — (0.1006)] 
= 1.3020 + 0.0667(1.0978) = 1.3752 (Corrector) (10.168) 


y(0.8) = y(0.4) +h 


Comparison would now be made between the values obtained from the 
corrector equation (10.168) and the predicted value from Eq. (10.167). 
Should these be sufficiently close, y(0.8) = 1.8752 would be retained as 
the value to use for further extensions of the range of solution. Had 
there been too great a difference between the two values, a second applica- 
tion of the corrector equation (10.163) would have been carried out. 
This time, however, y(0.8) = 1.8752 would have been used as the pre- 
dictor value. 

It is evident that Milne’s predictor-corrector method can be extended 
directly to the solution of sets of first-order differential equations. 
Assume, for example, that the two equations listed as (10.169) are 
prescribed. 

== fey) 1 = oud (10.169) 
Further, presume that four starting values, at equally spaced pivotal 
points, have been established by some other method. 


to Lo| Yo!|| fo} Jo 
ti(= to +h) | ail yi fil go 
to( = ti + h) | 22 Y2 || fe ge 
ts(= te + h) | x3 Y3 Il fs\ gs 


The predictor values are 
G4 = to + 446h(2f1 — fo + 2fs) (Predictor) (10.170) 
Ys = Yo + 48h(291 — ge + 293) (Predictor) (10.171) 
Correspondingly, the corrector equations are 
Le = to + VEA(fo + 4fs + fa) (Corrector) (10.172) 
Ys = Y2 + A(g2 + 493 + gu) (Corrector) (10.173) 


Milne’s method can also be extended to facilitate solution of equations 
of higher order. Consider, for example, the general second-order equation 


y”’ = f(z,y,y’) (10.174) 


As in the preceding cases, it is assumed that sufficiently accurate values 
of y, y’, and y”’ are available for at least four consecutive, equally spaced 
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pivotal locations. If these locations are denoted by the subscripts —3, 
—2, —1, and 0, the first derivative of y with respect to x at location +1 
can be predicted from the open-type equation 


Ya = ye + Hh Qy", — yl + 2y) (10.175) 
With this, a value for y,: can also be predicted. 
Yar = ya + MAG + 4y0 + yA) (10.176) 
A predicted value for the second derivative is 
yt = f(eyt yi") (10.177) 


A more nearly correct solution for y',, (and, in turn, y41) can be obtained 
by using the closed formula 


Ya = Ya t+ hy, + 4y0 + yt (10.178) 


The process would continue using Eqs. (10.176) to (10.178) until the 
desired accuracy had been achieved. 


10.8.2 Collocation Method 


The collocation method requires that the differential equation be satisfied 
to the degree of accuracy desired only at the pivotal points. It does not 
specify what is the situation between these pivotal locations, nor does it 
give an indication of the error in the integral. Nevertheless, consider the 
solution of the same first-order equation that was solved in the preceding 
section, using the difference expressions contained in Table 9.28. The 
given differential equation is 


eee aay (10.179) 
dz 


If a fourth-order parabola is assumed, the first derivative can be expressed 
in terms of five pivotal values (four known, and one being sought). From 
Table 9.28, 


il 
D(yo) = 24h (6y_4 — 3225 a7 72y_2 — V0U=t 4 5Oyo) Se Léhtyo’ 
(10.180) 


By selecting location 0 as the location of the desired “new solution,” the 
differential equation is written in terms of finite differences as shown in 
Eq. (10.181). 


[6y(0) — 32y(0.2) + 72y(0.4) — 96y(0.6) + 50y(0.8)] 


1 
(24) (0.2) 
= 1 — 0.8[y(0.8)] 
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or 


(6(1.0000) — 32(1.1775) + 72(1.3020) — 96(1.3672) + 50y(0.8)] 
= 1 — 0.8[y(0.8)] (10.181) 


(24) (0.2) 


This gives as an answer 
y(0.8) = 1.3745 (10.182) 


As a second example using the collocation method, consider the second- 
order differential equation 


*) jy OS (10.183) 
XL 


If a third-order system is assumed, the necessary backward-difference 
equations from Table 9.28 are 


d?y 1 

Jo? lee = a 6 U-8 + Ay-2 — Sya + 2yo) (10.184) 

dy 1 

late Ameer tale cae) (10.185) 
Yaa (10.186) 


Expansion about the unknown point x» has been assumed. 
Substituting the finite-difference expressions of Eqs. (10.184) and 
(10.185) into the differential equation (10.183) yields 


ii 
ip (UE om SS 1) 


1 
= £9 E (2 2yjee | Over vere 1146) | +yo+tC=0 (10.187) 
or 


yo(2 + t6xoh + h?) = y_s(1 + Maoh) — y_2(4 + 3420h) 


10.9 Stability of Numerical Solutions 
to Initial-value Problems 


Because of the general nature of this introductory text, only a minimum 
amount of information concerning errors has been included. From that 
given, it might be assumed that the error associated with a particular 
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calculated pivotal value is dependent only on the “local truncation error” 
of that calculational step. This is not necessarily true. In the solution 
of initial-value problems, particularly, round-off and truncation effects of 
previous calculations can markedly influence future errors, and when these 
build up rapidly, instability results. It should be noted that this type 
of numerical instability can occur even when the differential equation in 
question is inherently stable. 

Certain systems of equations and boundary conditions are inherently 
unstable, a given initial condition only partially (because of errors) sup- 
pressing an undesired major mode of solution. As the independent var- 
iable increases, the influence of these initial errors can completely domi- 
nate the solution. These ‘‘ill-conditioned”’ systems are characterized by 
the condition that a small change in the initial condition will result in 
a large change in the solution. 

For certain methods of solution, notably, the truncated Taylor series 
method and the Runge-Kutta method, instability occurs when h exceeds 
a certain limiting value. The particular limiting value depends on the 
equation being examined. However, no instability need be feared if h is 
kept ‘‘sufficiently small.”’ 

A third condition leading possibly to instability results from the appli- 
cation of a difference equation of higher order than that of the differential 
equation being examined. These higher-order difference equations may 
introduce “spurious solutions” that have nothing to do with the equation 
in question, and yet they may completely dominate the solution. Pre- 
dictor-corrector methods may result in this type of instability. It is to 
be noted, however, that here stability depends on how the process is used. 
If the iteration converges, and the process is continued until the corrector 
is ‘“‘exactly satisfied,’ the numerical process is stable. 


10.10 Boundary-value Problems 


As was pointed out earlier in this chapter, boundary-value problems are 
those for which constraints or conditions on the solution are specified at 
more than one value of the independent variable. Normally, these occur 
at “end points,”’ but this need not be the case. 

Conceptually, all the methods that were hereinbefore developed for the 
solution of initial-value problems can be used to solve boundary-value 
problems. It should be noted, however, that such direct extension to 
this class of problems normally requires a trial-and-error procedure start- 
ing from certain presumptions regarding the undefined initial conditions. 

The collocation method, which requires that the differential equation 
in difference form be satisfied only at the pivotal locations, is ideally suited 
to the solution of linear boundary-value problems. Consider, for exam- 
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Figure 10.9 


ple, the solution to the differential equation 
dx? te fale (10.189) 


The range of interest of the independent variable is0 <2 < A. The 
given boundary conditions are 
At az = 0: y=0 
(10.190) 

Atiww——4; y=0 
If it is assumed that a sufficiently accurate solution can be obtained by 
subdividing the range in question into five equal parts, that is, h = 144A, 
the problem can be described as shown in Fig. 10.9. The desired func- 
tion values at x1, Ye, x3, and x4 are, respectively, yi, Yo, yz, and ys. 

Irom Table 9.27, by assuming a second-order system with symmetrical 
placement of pivotal points on either side of the location in question, 72, the 
second derivative of the function can be approximated by the expression 


1 
D*(yi) = h2 (yi — 2ys + yes) (10.191) 
In difference form, then, the given differential equation (10.189) is 


il 
h? (Yi-1 20 Yit1) se fi(xi) = 0 
or (ya — 2y; +’ yig1) + h?fi(z,) = 0 (10.192) 


At location @)—where z; = 21, yi-1 = Yo, Yi = Y1, aNd Yin1 = Y2—the nec- 
essary condition is | 


Yo — 2y1 + yo + hfi(ai1) = 0 (10.193) 


But the given boundary condition y(0) = 0 requires that yo = 0. There- 
fore Eq. (10.193) can be written as 


—2y1 + yo + h*fi(ai) = 0 (10.194) 


numerical solution of ordinary differential equations 289 


Expansion about each of the other pivotal locations will result in a similar 
linear algebraic equation. The total set is given by 


—2 1 0 Ol] wy —h*f,(x1) 

oN 1 Ol] ye —h?f;(x2) 
Ne = (10.195) 

0 1 TS. Ys —h?f,(xs) 

0 O ONG Y4 —hafi(va) 


Solution can be achieved by any of the methods defined in Chap. 5. 

The reduction of the original differential equation to a set of n linear 
algebraic equations is characteristic of solutions of linear boundary-value 
problems using the collocation method. The unknowns are the function 
values at the pivotal locations. 

Had the differential equation in question been of the fourth order, it 
would have been necessary to express the fourth derivative in terms of 
five adjacent pivotal locations. Expansion about location @ leads to the 
condition illustrated in Fig. 10.10. Note that location 7 — 2 falls com- 
pletely outside the range of variable of interest. More than this, had it 
been necessary to ensure that the differential equation (in difference form) 
was satisfied at location 0, both 7 — 2 and 7 — 1 would have been ‘“‘off.”’ 
One method for handling such situations is illustrated in Fig. 10.11. 
Expansion about location 0 presumes a difference equation written in 
terms of pivotal values at 0 and forward of it. For the second case, 
expansion about location 1, it has been presumed that equations are avail- 
able which allow definition of derivatives about a location one removed 
from the leftmost pivotal point—so-called ‘‘off-center’” forward-difference 
expressions. The third, fourth, and fifth cases illustrated would be writ- 
ten in terms of pivotal values symmetrically placed with respect to the 
reference locations. 

A second—more often used—method for handling problems of this type 
is to first write all equations in symmetrical form, presuming a fictitious 


Figure 10.10 
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Figure 10.11 


range of independent variable with corresponding unknown pivotal values 
at fictitious locations. The relationships of the pivotal values in the fic- 
titious range to those in the range of interest would be defined by the 
specified boundary conditions of the problem in question. 

To illustrate how such a procedure works, assume that a fourth-order 
system is specified, and that boundary conditions at 0 are defined as 
follows: 


Youd (10.196) 
D(yo) = 0 (10.197) 
D*(yo) = 0 (10.198) 


The pivotal values in question are shown in Fig. 10.12, as is the assumed 
fictitious range. From Table 9.27, 


1 
D(yo) = 1h (y-2 — 8y-1 + 8y41 — Y42) (10.199) 
A 1 
D*(yo) = Oh} (—y-o + 2y-1 — Qyar + y42) (10.200) 
Figure 10.12 
iy 
ae 
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Substituting these into the specified boundary conditions gives three 
simultaneous equations in the unknowns y_», y_1, and yp. 


een | 
y-2 — 8y1 + 8y41 — Yr2 = 0 (10.201) 
wU-2 te 2Ya1 — 21 1 Ye = 0 
Solution of this set of equations requires that y_» = y42, y-1 = Y41, and 
Yo = q. 
It is to be noted that boundary conditions are not always related to 


only one of the derivatives. For example, the fourth boundary condition 
specified in Example 7.4 is 


yl" a8 k2y’ a0 


or D3(yo) + k? D(yo) = 0 (10.202) 
For this case the corresponding difference expression is 

1 k? 
ays 62H Aya = yar + Ys) oe ye — 8y-1 + 8y41 — Y42) = 0 

(10.203) 

or 

ena) —y(2— Sait tale — Sa) Yel — a) = 0 (10-204) 

Kh? 

where 0 S25 (10.205) 


The two most often encountered pairs of boundary conditions are 


ie oY 2% (10.206) 
dx 
d*y 
and y=0 are 0 (10.207) 


The first of these is illustrated in Fig. 10.13. It is to be noted that solu- 


Figure 10.13 
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tion requires the following relationships: 
Jo 0 Y-1 = Y+1 U2 = pe (10.208) 


For the second set of boundary conditions—y = 0 and d?y/dx* = 0— 
the relationships illustrated in Fig. 10.14 would be required. 

To illustrate the application of these procedures, consider the solution 
to the following fourth-order differential equation: 


4 
x Xx 


d 

Ata = 0: = Oe and, men e=n0 (10.210) 
dx 
d?y 

Ata = "A: y= 30) and —=0 (10.211) 
da? 


It is to be assumed that the desired accuracy can be achieved by sub- 
dividing the range of x of interest into five equal parts, as shown in Fig. 
10.15. For completeness, the two fictitious regions, one at each end of 


Figure 10.15 
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the range of interest, have also been indicated. The difference equation 
corresponding to the given differential equation (10.209) is 


1 
ja (Yina — Aya + By — 4yees + irs) 
1 
+ C155 (yia — 2ys + yess) + C2 = 0 (10.212) 


or 


yi-2(+1) + yis(Cih? — 4) + yi(—2Cih? + 6) + yayr(Cih? — 4) 
+ yix2(+1) = —C2h* (10.213) 


It is to be noted that in Eq. (10.212) a fourth-order (five points) 
polynomial is used to express the fourth derivative, whereas a second-order 
(three points) polynomial is used for the second derivative. This was 
done to keep the order of the error terms the same (both are of the order 
of h?). 

The boundary conditions require that pivotal values at the fictitious 
locations be related to those in the range in question according to the 
following: 


Yar — Yui and Yeti Yen (10.214) 


The governing matrix is therefore 


ett Cie — A +1 0 
Cede 0 6) On — 4 +1 
+1 Ch Ae OC ce eee a7 it 
0 +1 Che ee = 20 
—Coht 
= hoe) (Ue 
—Ch4 


The solution procedure just demonstrated—where, by using the bound- 
ary conditions, fictitious pivotal values are first defined in terms of values 
within the range in question—is not the only procedure available. The 
problem could equally well have been formulated in terms of eight 
unknown pivotal values: y-1, Yo, Y+1, Y+2, Y+s, Yrs Yrs, and yzs. The 
four additional equations would correspond to the specified boundary 
conditions. 
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It is to be noted that this matrix is not symmetrical. 

The collocation method can be equally well adapted to the solution 
of sets of boundary-value differential-equation problems. Consider, for 
example, the two equations 


dx dy 
pete i ee oa 2 
pee a 8 (10.217) 
d?y 
dt? & (10.218) 
The given boundary conditions are 


If it is assumed that a sufficiently accurate solution can be obtained by 
dividing the range of independent variable in question into four equal 
parts, the unknown pivotal values are those shown in Fig. 10.16. Writing 
the first of the two differential equations in symmetrical difference form, 
assuming a second-order approximating polynomial, yields the following 
difference equation: 


if 
i (tia — 205 + rig) + WA(—Yyi- + Yui) — 382; = 13 (10.220) 


or 


Gea) (+1) P22 8h’) -F ri (+1) 
+ yi(—WA) + Yri(+34h) = +18h? (10.221) 


Figure 10.16 
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For Eq. (10.218), the corresponding difference equation is 
a(—h®) + ysa(+1) + y(—2) + yai(t+) =0 (10.222) 


The equations controlling the solution are 


—2 — 3h? +1 0 0 +h 0 L1 
+1 —2 — 3h? +1 — Wh 0 +h || xe 
1) +1 —2 — 3h? 0 —h 0 L3 
—h? 0 0 —2 ae 0 Yi 
0 —h? ) =; Il i sit Y2 
0 0 —h? 0 +1 —2 Y3 
+13h? 
+13h? 
2 
= ve °1 (10.223) 
0 
0 


10.11 Eigenvalue Problems 


Eigenvalue problems (or characteristic-value problems) can also be solved 
by using the collocation method just described. Consider, for example, 
the solution to Eqs. (10.224) and (10.225). 


d? 
Be Sy Et (10.224) 
Geb 
Aiea O° y=0 
(10.225) 
Atz = L: y= 0 


By recalling that the symmetrical difference expression for the second 
derivative is 


1 
he Ga 20: A Uisa) 
the differential equation can be written in difference form as 
1 
. (yi-v — 2yi + yeu) + Vy: = 0 


or Yer — 24; + yen Oh a =O 
or yi-(+1) + yi(—2 + 7h) + yrur(+1) = 0 (10.226) 


If the range of independent variable in question (that is, L) is divided 


into two segments (n = 2), the problem corresponds to that illustrated 
in Fig. 10.17. 


I 


l| 
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Writing Eq. (10.226) about location @ yields 
yo( +1) + ys(—2 + A*h2) + y2x(+1) = 0 
But yo = yz = 0. Therefore, 
yi(—2 + Ah?) = 0 (10.227) 


For a nontrivial solution, it is necessary that 
—2 + 7h? = 0 
The eigenvalue solution for this one-degree-of-freedom assumption is, 
therefore, 
2 8 


2 = 
1 Se 


Soe (10.228) 


If the region is subdivided into three segments (n = 3) (see Fig. 10.18), 
solution proceeds in the following fashion: 
Expansion about location @) gives 


y1(—2 + APA?) + y2x(+1) = 0 (10.229) 


Expansion about location () gives 


Gy Ml — 2 A it) 0) (10.230) 
Figure 10.18 
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In matrix form, the controlling set of equations is 


—— 
—2 + d*h? =F ll Yi 0 
= (10.231) 
= —2 + n*h? Y2 0 
—— 
Using the methods of Chap. 7 gives the smallest eigenvalue as 
Nias = = (10.232) 


Er? 


The governing set of equations for the case where n = 4 (that is, 
h = L/A4) is 


—2 + 2A? ay 0 Y1 


+1 —2 + dh? +1 y2 | = [0] 
——- 


0 stad —2 + dh? Ys 
= 
(10.233) 
Here, the smallest eigenvalue is found to be 
9.36 
Kar= 7 (10.234) 
and the corresponding eigenvector is 
+0.707 
+1.000 (10.235) 
+0.707 


For an indication of the error associated with each of these solutions, 
consider the nondimensional plot of Fig. 10.19. Note that both of the 
coordinate dimensions are ‘“‘squared.”’ 

As a second example, consider the solution of the more involved elgen- 
value problem defined by Eqs. (10.236) to (10.238). 


d‘y ay 
sec egal (10.236) 
At 2 = 0: y= 0 and 4 "= 0 (10.237) 


Atay =—L: Yee O and yl! + ky’ = 0 (10.238) 
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Figure 10.19 


It is assumed that sufficient accuracy can be realized by selecting n = 4. 
The problem is defined graphically in Fig. 10.20. Two fictitious ranges 
have been presumed: one to the left of location ©, the other to the right 
of location @. 

From Table 9.27, the following difference expressions for derivatives 
are obtained: 


1 \ 
D3(y:) = — (—yi-2 + 2yi-1 — yi + yrs) (10.239) 


Bhs 
1 

Dh) ie Gea = 20 Yer) (10.240) 
1 

D(yi) = oh (ert ers) (10.241) 


To establish the relationships in the fictitious region to the left, consider 
boundary conditions (10.237): y(0) = 0 and y’/(0) = 0. These give 


Niles AU Ve a (10.242) 


Figure 10.20 


Fictitious Fictitious 
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For ys and ye, it is necessary to consider the constraints y’"(L) = 0, and 
y''(L) + k*y'(L) = 0. Using the first of these, 


0 


1 
he (ys = 2Y4 + Ys) 


or Ys = 2y4 — Y3 (10.243) 
For the second boundary condition, y’’’(L) + k?y’(L) = 0, the relation- 
ship is 
k?2 
ape +? + 2yn= 2Y 5 Ye) oh (—ys + ys) = 0 
or Yo = Yo +1) + y3(—4 + 2a) + ys(4 — 2a) (10.244) 
where ake 


The difference equation corresponding to the given fourth-order dif- 
ferential equation is developed as follows: 


il k? 
hé (Yi_2 = 4yi-1 == by: nae AYi41 et 2) a he (Yer SLT ES = Yiti) = 0 
(10.245) 


or Yyiio(+1) + ya(—4 + a) + yi(6 — 2a) + yi (—4 + a) 
+ yinx+1) =0 (10.246) 


Writing Eq. (10.246) at each of the pivotal locations 1, 2, 3, and 4, 
taking into account the previously defined relationships for the fictitious 
pivotal points, yields the desired four simultaneous linear algebraic homo- 
geneous equations. 


7—2a —-4+ a +1 0 Yi 
—4+a 6—-2a —-4+a +1 Y2 = [0 
ap ll —4+a 5-—2a@ -2+al|| ys} 
0 +2 ° -4+a 2— 2a Ys 
(10.247) 
The smallest eigenvalue associated with this 4 X 4 set is 
An—4 = 0.0745 (10.248) 


Since a = hk?, this solution can equally well be expressed in terms of kL. 


ii; 2 
a = h*k? = (*) k? or (KL)? = an? (10.249) 
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where n is the number of equal-length segments into which the interval L 
is subdivided. Therefore, 


(kL)?_, = (0.0745) (4)2 = 1.192 (10.250) 


This should be compared with the exact solution 


(KL)? sack = 2.475 (10.251) 


It could be speculated that the large error associated with solution 
(10.250) is due to the dominating influence of the boundary conditions in 
the selected four-variable system. If eight segments are chosen, the first 
eigenvalue is 

(kKL)?_, = (0.0384) (8)? = 2.460 (10.252) 


Table 10.14 summarizes the first three eigenvalues obtained in the man- 
ner described above, assuming both n = 4 and n = 8. For comparison, 
exact values have also been listed. 


Table 10.14 
n First eigenvalue Second eigenvalue Third eigenvalue 
4 (KL)? = 1.19 (kL)? = 16.7 (kL)? = 44.2 
8 (KL)? = 2.46 (Us) 732186 (kL)? = 57.0 
Exact value (KL)? = 2.48 (KL)? = 22.3 (kL)? = 61.9 


10.12, Nonlinear Boundary-value Problems 


It is evident from the preceding two sections of this chapter that the col-, 
location method is ideally suited to the solution of linear boundary-value 
problems. Had the differential equations in question been ‘nonlinear, 
there would have resulted sets of nonlinear algebraic equations. 

For single nonlinear boundary-value problems, a trial-and-error initial- 
value approach, using the methods described earlier in this chapter, is 
more often used. As a demonstration of this general approach, consider 
the following differential equation and boundary conditions: 


2 
he ry? (10.253) 
Ey 
y(0O) = 1.0 and y(1.0) = 1.0 (10.254) 


It is noted that y’(0) is initially unknown. Therefore, to be able to use 
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any of the initial-value procedures, it is necessary to first assume start- 
ing values for this derivative. The correct selection of y’(0) results in 
aL) = 0, 

In the example, since it is being presented only for illustrative pur- 
poses, the Runge-Kutta equations listed in Table 10.10 have been used 
for numerical integration throughout the entire range0 <2 < 1.0. An 
interval n = 0.2 has been presupposed as resulting in a “‘sufficiently accu- 
rate’ solution even though such an assumption may be questioned. 
Function values are given in Table 10.15 for several different starting 
values for y’/(0). Corresponding curves are shown in Fig. 10.21. The 
correct starting value is y’(0) = —0.425. 


Tablel0.169 yo 470) =. 07 G0) =e 0 


Function Assuming Assuming Assuming Assuming 
value y'(0) = 0 y’(0) = —1.0 y'(0) = —0.6 y'O) = —0.425 
yO) = 1.0000 1.0000 1.0000 1.0000 

y(0.2) = 1.0201 0.8176 0.9188 0.9340 

y(0.4) = 1.0821 0.6623 0.8716 0.9030 

AO) = 1.1914 0.5248 0.8550 0.9048 

y(0.8) = 1.3581 0.3984 0.8679 0.9396 

yi(e0) = 1.5996 0.2785 0.9111 1.0000 


_—_—_—_————— 


Figure 10.21 
2.0 - 2 


Assuming y'(O) 


Calculated y(1.0) 


e) ee ee eee Re ee ores orto Fs 
(0) 0.5 eC) =| (0) =(0)5) O 


x Assumed y (0) 
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In general, for nonlinear boundary-value problems, superposition does 
not hold. ‘This is the reason for the trial-and-error approach used above. 
However, if the problem in question is a linear one, or even if it consists 
of a series of linear problems, this is not necessary, and superposition of 
any two independent solutions allows the matching of the two specified 
boundary conditions. For example, if the solution corresponding to a 
case A is denoted as y(2) case 4, and that corresponding to case B is (2) case B, 
then a given boundary value y(a) at location a can be realized by requir- 
ing that 


C1y(@) case + Coy(G) case Ba y(a) (10.255) 
Similarly, at location 6, the other specified boundary location, 
7G 0) canout Si Cay (b) case B = y(b) (10.256) 


Casting these in matrix form yields 
rs ee (10.257) 


aes A WD) gee | ey 

AOR A y(b) case B Cy y(b) 

With values obtained for C; and C; from (10.257), the final solution over 
the entire range of interest is 


y(x) = Ci) thse A as Cou) cose B (10.258) 


Nonlinear problems can be reduced to linear ones in a variety of ways, 
the accuracy of solution depending on the method of reduction used, the 
numerical-integration process selected, and the interval of integration 
chosen. For example, in the problem just solved by the trial-and-error 
approach, it can be assumed that the given nonlinear equation 


y =o 20 <2<1.0) (10.259) 
can be represented by a series of five linear differential equations of the 
type 

USI (10.260) 
where y; is presumed to be constant over each of the intervals in question 


and has a magnitude equal to the beginning value of y in that interval, as 
shown in the following table. 


Range of solution Differential equation 
First interval OSS 012 a” = nO 
Second interval OPiS tess O4! a! =o OANE 
Third interval 0.4<2< 0.6 y”’ = [y(0.4)]? 
Fourth interval 0.657508 Oe! = OLE 
Fifth interval Oe) Same < ILO af” = lke 


a 
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With the reduction assumption and the following selected two independ- 
ent initial values, 


Case A: y(0) = 1.0000 y (0) = 0 
Case B: y(O)r=0 y'(0) = 1.0000 


the Runge-Kutta integration solutions shown in Table 10.16 are obtained. 
Here, again, it is to be recognized that the question of the applicability 
(or desirability) of the Runge-Kutta method as contrasted with, for exam- 
ple, the predictor-corrector approach over the entire range 0 < x < 1.0 
has not been considered. 


(10.261) 


Table 10.16 
Case A Case B 

x y (x) y' (x) y(z) y' (x) 

0 1.0000 0 0 1.0000 
OR2 1.0200 0.2000 0.2000 1.0000 
0.4 1.0808 0.4081 0.4008 1.0080 
0.6 1.1858 0.6417 0.6056 1.0401 
0.8 1.3422 0.9229 0.8209 1.1135 
10) 1.5628 2S a2 RO Siva 1.2483 


To satisfy the boundary conditions listed in Eqs. (10.261), it is required 


that 
1.0000 0 C,}| _ | 1.0 
foes Aiea ea a Hey hers 
This gives Cy; = 1.0000 and C, = —0.5324. The solution y(x) corre- 
sponding to this reduction assumption is, therefore, 


y(2) aa 1.0000[y (2) case 4] cal 0.5324[y(X) case B] (10.262a) 


The final values are shown in Table 10.17. Also listed are the values 
previously obtained by the trial-and-error approach. (It is to be noted 
that the Runge-Kutta ua: listed in Table 10.10 were used for both 
sets of integrations. ) 

There are ways other than that illustrated above to linearize nonlinear 
problems. One, based on a generalization of the Newton-Raphson tech- 
nique for obtaining roots of nonlinear algebraic equations, f has proved to 

+R. McGill and P. Kenneth, A Convergence Theorem on the Iterative Solution of 

Non-linear Two-point Boundary-value Problems, XIV Intern. Astronaut. Federation 

Congr., Paris, September, 1963, pp. 173-188. 


R. McGill and P. Kenneth, Solution of Variational Problems by Means of a Generalized 
Newton-Raphson Operator, AJTAA J., vol. 2, no. 10, pp. 1761-1766, October, 1964. 
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Table 10.17 
a 
Solution of Solution of 
linear problem nonlinear problem 

x [Eq. (10.262a)] (Table 10.15) 
0 1.0000 1.0000 

O72 0.9135 0.9340 

0.4 0.8674 0.9030 

0.6 0.8634 0.9048 

0.8 0.9052 0.9396 

1.0 1.0000 1.0000 


be adaptable to a large variety of nonlinear differential-equation prob- 
lems. In general, it is an iterative method. For a two-variable system, 
one independent and one dependent, again considering the second-order 
equation examined above [y’’ = f(z,y)], the linearization is according to 
the following expression: 
Given nonlinear equation: 

y = F2,y) (10.26 3) 


Corresponding linear equation: 


an () OR ee roncioag ot) (10.264) 


It is to be noted that Eq. (10.264), in the rearranged form 


(nt1y)”” — nf(x,y) 
n(Of/dy) 


corresponds to the first-order Newton equation (8.45). However, instead 
of convergence to a zero solution, as required in that case, convergence to 
(n+1y)”’ is here presumed. 

_ In both equations (10.264) and (10.265), the pre-subscripts indicate 
iterations. It is assumed, at any one stage of integration, that the n pre- 
scripted terms are known (that is, they are, for any particular integra- 
tions, constants) and that the variables in the desired differential equation 
are the n + 1 pre-subscripted terms. The differential equation is 


7) rs) 
(n4iy)’” = nti | (| — ny | (4) + nf(x,y) (10.266) 


Constant Constant 


Consider, for illustration, the formulation of the linear system correspond- 
ing to the nonlinear problem just discussed. : 
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y” =f(x,y) =y¥? (10.267) 
F) 

Since a = 2y (10.268) 
dy 


the equation corresponding to Eq. (10.266) is 
(nay) = ngr¥l2(ny)] — ny[2(ny)] + (ny)? 
or Cnty)” = n4il2(ny)] — (ny)? (10.269) 


If it is assumed as a starting situation that oy(#:) = 1.0000, x; = 0, 0.2, 
0.4, 0.6, 0.8, and 1.0, the linear differential equation (in all ranges) is 


Gy)’ = 1y(2) — 1.0000 (10.270) 


For assumed initial values y(0) = +1.0000 and y’(0) = 0, integration 
using the Runge-Kutta equations of Table 10.10 results in the “new” 
values for 1y(x) shown in the following table. 


zr oy (x) 1y (x) 
0 1.0000 1.0000 
0.2 1.0000 1.0201 
0.4 1.0000 1.0821 
0.6 1.0000 1.1910 
0.8 1.0000 1.3555 
150 1.0000 1.5890 


It is to be recognized that the linear differential equations for the next 
iteration are different for each range of interest. 


(oy)(1) — 1.0000 for 0<2<02 
(oy)(1.0201) — 1.0406 for O02<2<04 
(oy)” = < (ey) (1.0821) — 1.1709 for 04<2<06 (10.271) 
(y)(1.1910) — 1.4185 for 06<2<08 
(sy) (1.8555) — 1.8873 for 08<2<10 


Solutions for several iterations are given in Table 10.18. 


For a second independent case, where it is required that the initial 
conditions be 


y(0) = 0 and y’(0) = +1.0000 
the solution is that given in Table 10.19. 
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Table 10.18 
eee 
x oy (x) 1y (x) 2y (2) ay (2) 
————— ee 

0 1.0000 1.0000 1.0000 1.0000 
0.2 1.0000 1.0201 1.0201 1.0201 
0.4 1.0000 1.0821 1.0821 1.0821 
0.6 1.0000 1.1910 1.1914 1.1914 
0.8 1.0000 R300 1.3579 1.3579 
1.0 1.0000 1.5890 1.5987 1.5987 


Table 10.19 

x oy (x) 1y (x) 2y (x) 3y (z) 
0 0 0 0 0 
0.2 0.2000 0.2000 0.2000 0.2000 
0.4 0.4000 0.4013 0.4013 0.4013 
0.6 0.6000 0.6088 0.6088 0.6088 
0.8 0.8000 0.8309 0.8310 0.8310 
1.0 1.0000 1.0803 1.0867 1.0867 


Given the information defined in Tables 10.18 and 10.19, solution to 
any given boundary-value situation proceeds as defined earlier, according 
to Eqs. (10.257) and (10.258). 

It is to be observed that in both of the cases examined, convergence is 
rapid. This, however, is not an isolated occurrence; rather, it is a char- 
acteristic of the procedure. In discussing the convergence of the method, 
McGill and Kenneth state the following: ‘‘The technique discussed herein 
is systematic and rapidly convergent, and owing to the quadratic con- 
vergence, great care is not required in obtaining starting functions. It 
is sufficient that the trial function be not so bad as to cause divergence.”’ f 

It is evident that the iterative scheme just described can be extended 
to the solution of other types of equations. For example, if the second 
derivative is defined to be also a function of y’ the equation is: 

Given nonlinear equation: 


y” = f(x,y,y’) (10.272) 


+R. McGill and P. Kenneth, A Convergence Theorem on the Iterative Solution of 
Non-linear Two-point Boundary-value Problems, XIV Intern. Astronaut. Federation 


Congr., Paris, September, 1963, p. 185. 
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Corresponding linear equation: 


0 ip 
coy” = (2) Gousy = aad + (S5) Goats! — 0!) + ov) 
(10.273) 


If the equation is a nonlinear first-order equation, the iterative scheme is 
defined by Eq. (10.275). 


Given nonlinear equation: 
y’ = f(a,y) (10.274) 


Corresponding linear equation: 


(ngiy)’ = (3) (n4ay — wn) + nf (ZY) (10.275) 


The generalized Newton-Raphson operator approach has a further 
advantage. It can be readily extended to the solution of sets of non- 
linear problems. For illustration of how this might be accomplished, it 
is assumed that all the equations in question are of the first order, and 
that the independent variable is time ¢. The dependent variables are 
21,22, X3, . . . , en, and differentiation with respect to time is denoted by 
a dot over the variable in question. 

In matrix form, the set of equations under consideration is given by 


[Xi (Fea (10.276) 

where X= a os, 
Ee =f fo, fase oe ee (10.277) 

ts = I5(Gi,23,03, oe eee) 


The range of independent variable of interest is 
ESTE (10.278) 


For this system, N boundary conditions are specified, part at ¢ = t, and 
the remainder at &. For illustration, it is assumed that half are specified 
at ¢2 and the remaining 14N are specified at t,; but this need not be true. 
The specific values are defined as follows: 


Li(ta) = Lia til) = te 
ey(ta) = "tea" Lol) = Lop 
(10.279) 
‘tw jo(ta) = X(N /2)a tn j2(t) = X(N/2)b 


The linear iterative differential equation corresponding to Eqs. (10.276) is 
[n41X] = [J (Xt) [narX — .X] + LF (X,0)] (10.280) 
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where [/(X,t)] is the jacobian matrix of partial derivatives of f; with 
LESPECE MOM; ¢ = 1, 2, ag Neg = 1,2... 2 oN 


Of 
i) (10.281) 


For a 4 X 4 set, the expanded equations are: 
Given nonlinear equations: 


x1 = fii(21,%2,03,X4,t) 
bo = fo(x1,%2,03,04,t) 

(10.282) 
t3 = f3(@1,%2,03,04,t) 


tq = fa(1,%2,%3,04,t) 


Corresponding linear equations: 


leet 

(2) 0X2] n\O%3/_— n\OX4 
n41%1 (2) (#) (2) ($4) nt101 — nX1 
n+1%2 0X1 0X2) n\OX2/- n\OX4 nt1l2 — n&e2 
n4+143 = Ofs Ofs Ofs (=) Peta, == 105) 
a) HCD) (Ee ce] bere 

=) ded enh les 

ie n\OX2F  n\OX3/ n\OX4 


nf 2(@1,%2)23,L4,t) 

10.283 
nf 3(21,X2,03,04,t) ( ) 
nfs(£1,09,03,04,8) 


Emre 


The first of these linear expressions is 


ayiti = (3) (n4101 = Ata) a7 (32) (n41%2 — pX2) 


0x4 n 0X2 
te) 
+ (3) (n41%3 == ns) + (2) ate a ns) = nfi1(1,€2,03,04,t) 
n 0X3 n 0X4 
(10.284) 


It is to be remembered that the n pre-subscripted terms represent, for any 
one cycle of iteration, constants. 
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The boundary conditions for the system of equations under considera- 
tion are, for example, 
Li(ta) = Lia £1(t) = x1b 


(10.285) 
tod 


ll 


and Lo(ta) = Lea Lo(ts) 


This number of constraints is consistent with the prescribed four-depend- 
ent-variable first-order systems. Correspondingly, to be able to realize 
these specified values 210, 2a, 21, and x» by using the method of super- 
position, four independent solutions must be obtained. In selecting these 
independent cases it is to be recognized that certain assumptions for the 
initial conditions can facilitate the numerical process of solving the result- 
ing linear algebraic equations. Precision can also be improved by judi- 
cious selection of these values. 

It is to be recognized that the linear differential equations (10.283) are 
equivalent to 


[X@] = [(AMIX(] + (BO) (10.286) 


However, not all cases need be solved by using this full equation. If, for 
example, only the influence of x3(t.) were desired, it would be sufficient to 
examine the solution of the homogeneous equation 


[XQ] = [AOMIXO] (10.287) 


For the four-variable system, then, initial conditions, as shown in Figs. 
10.22 to 10.25, might be specified. 


Figure 10.22 
Ft 
CasE I ig 
L1(ta)r = Via ne 
Lo(ta)1 = O Xa Saw = x11 
L aL =e } 
X3(ta)t = ft f e 
La(ta)r = O | | | 
X2 61 
ll | ft Xo 
| 
oes j 
x3 
oa ee 
ty | ir 
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- 
Case II Fe 


ri(ta)rr = O mA 
x 
NU en = oy. | i | as 
=I ah x4 
Za(ta)ir =O oy 
ta(fa)r1 = O | -X2b3 
iE a J K Xo 
x3 
| 
ak 1 J x4 
fa ty 


Figure 10.23 


Under these assumptions, the algebraic equations to be solved to allow 
matching of the boundary conditions are 


Via 0 0 0 Ca Via 
OP tea 0 0 C2 Loa 


= (10.288) 
Lib1 = L1b2 = iv3 1b4 C3 Lib 
C21 =L2n2 XLov3 + Lo24 C, Lo 
Cask ITI Figure 10.24 
Ci(ta)irx =" OU ae 
Xeo(ta)111 = O eee 
| | x4 b2 
X3(ta)111 = 1.0 — : |_____ ae x 
Za(ta)i1r1 = O Y \ 
X2a les | | a eh X2b2 ra 


| as x4 
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CasE IV —: 
ti(ta)iv = O 
= fae 
talla)iv = 0 | | Mba : 
23(ta)tv = 0 K : 
eV Ae 
La(ta)iv = 1.0 H X2b4 
1 H i } i Xo 
! ; 
| i ey x3 
a 
et ! 
Hi L u | J x4 
tq tp 


Figure 10.25 


The C’s having been determined, the final expression for z;(t) is 


a(t) = Ci(a)y<t Col) + Ce(@ainr + Calziiv (i = 1, 2, 3, 4) 


(10.289) 

For a numerical example, consider again the solution to the system 
NE fea Ye (10.290) 

and y(0) = 1.0 and y(1.0) = 1.0 (10.291) 


By substituting z = y’, the second-order nonlinear equation can be reduced 
to two first-order equations: 


yf = 2 (10.292) 
a =u" (10.293) 
The boundary conditions remain the same as those listed in Eq. (10.291). 
The first-order linear iteration formula corresponding to Eq. (10.292) is 
nt1Y’ = U(ngie — nz) + nt = nite (10.294) 

The second equation, corresponding to Eq. (10.293), is 


np = 2 nY(ng1y — a) + ny? = 2(ny)(n4ry) — (ny)? (10.295) 


By using the Runge-Kutta equations of Table 10.8, assuming as initial 
conditions y(0) =.1.0 and 2(0) = 0, the solutions listed in Table 10.20 
were obtained. It should be noted that because of the particular nature 


of the linear equation (10.292), it was not necessary to make an initial 
assumption for z(z). 
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Table 10.20 y' =z, 2' = y?, y(0) = 1.0, 2(0) = 0 
ee ee eee ee 


Function 

values Initial assumption First iteration Second iteration Third iteration 
ee ee ee a 

y(0) = 1.0000 1.0000 1.0000 1.0000 
YO) = 1.0000 1.0201 1.0201 1.0201 
y(0.4) = 1.0000 1.0821 1.0821 1.0821 
y(0.6) = 1.0000 1.1910 1.1913 1.1913 
y(0.8) = 1.0000 1.3556 ae 1esout 
y(1.0) = 1.0000 1.5891 1.5893 1.5893 

A) = 0 0 0 
Zz Or2) 0.2027 0.2027 0.2027 
2(0.4) = 0.4217 0.4220 0.4220 
2(0.6) = 0.6746 0.6780 0.6780 
2(0.8) = 0.9819 0.9990 0.9989 
Cle) — 1.3682 1.4290 1.4290 


Prob. 10.1: Using at least two of the methods discussed in this 
chapter, solve the following initial-value problems, assuming h = 0.2: 
aij — oy a0) = 1.0507 = 1.0 
(Det = li ys (0) 221.0. (0) =" 1.090 a= 0 
(c) (y)(y’) +1 = 0; yO) = 0, (0) = 10;0< 2 < 2.0 
(d) y’ + Asin y = 0; y(0) =a, y'(0) =0;0 <2 < 1.0; 
where:A = 2:0) ai=10:5 
(e) 6 + 20+ 0.16? = 10; 00) =0;0 < 6@< 1.0 
(f) 2’ + 40(1 + cos x)z = cos 3x; 2(0) = 0, 2’(0) =0;0 <x < 1.0 
(g) €+0.1z + 1/2? = 0; x(0) = 1.0, (0) = 10.0;0 < ¢ < 2.0 
(h) F”’ = F’ — FF’ + 4; F(O) = 0, F’(0) =0;0 <2 < 1.0 
(i) B’’ — ?B’+ B+ C =0; B’(0) = 0, B’(O) = 1.0, BO) = 1.0; 
0<2z< 1.0; C = 3.0 
Prob. 10.2: To three significant figures, determine the solution to the 
following sets of differential equations: 


ee ee 0) 0-90) = 1.070 <4. 05 


Up IEA a) 
O) FZ FUF! 4) = 0, yO) = 1.0, #0) = 0;0 St < 10 


y 

(ce) ¢=x2+2y+1 
y=yt2z+t 2(0) = 0, y() = 1.0, 2) = 1.0;0<'< 1.0 
z2=2+2r+t 


@) oa x(0) = 1.0, £(0) = 0, y(0) =0;0 <t< 1.0 
y -—J 
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Prob. 10.3: Using the predictor-corrector method of Milne, extend 
the range of solution of Table 10.13 for the equation y’ = 1 — zy, from 
that given (x = 0, 0.2, 0.4, and 0.6) to x = 2.0. Use an interval of 
h = 0.2. 

Prob. 10.4: For the given set of equations 


t= oL 


io) x(0) = 1.0, yO) = —1.0 


obtain starting values at t = 0, 0.2, 0.4, and 0.6. Then, using the Milne 
predictor-corrector method, extend the range of solution tot = 2.0. The 
increment should be selected as h = 0.2, and four significant figures are 
desired. 

Prob. 10.5: Determine values of y at four intermediate pivotal loca- 
tions in the range 0 < x < 1.0, for each of the following boundary-value 
problems: 

(a) y”’ + 4y = 22 + 1; y(0) = 0, y(1.0) = 1.0 

(ob) yf! = 4y = &; yO) = 0; 70.75) — 10 

(ce) yl) hy! 4 = 0; 90) = 0.9) 0) = 00) 07 

(d) y”’ — Cy’ +4 = 0; y(0) = 1.0, y(1.0) = 0 

Prob. 10.6: For the homogeneous equation y’’ + \*y = 0, determine 
the first two eigenvalues, when the boundary conditions are 

(a) y(0) = 0, y(1.0) = 0 

(0) yO) = 0; y7-0)"= 0 

(c) yO) = 0, y’(1.0) = 0 

(d) (0) = 0, y’(1.0) = 0 

Prob. 10.7: An elastically supported, uniform cross-section column 
of length L, fixed at one end and pin-connected at the other, is subjected 
to an axial thrust P, as shown in Fig. 10.26. The governing differential 
equation is 

Coie Ne) k 

wees ————y=0 

CE MONET eee NINE 
where k is the spring constant of the elastic support (force per unit of 
deflection per unit of length along the member), and EI represents the 


ie 
— | 


Figure 10.26 
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bending stiffness per unit of length of the cross section. The boundary 
conditions are 


y(0) = 0 y'(0) =0 
y= 0) y(L).= 0 


To change the range of integration, it is assumed that a new independent 
variable z is introduced, where 


The resulting equation is 


dy PLA dy BLA _ 
dz* EI dz? Ie! 


The corresponding boundary conditions are 
790 (0) a= 0) 
y(1.0) = 0 y’’(1.0) = 0 


(a) For the case where y = kL4/PL? = 2.5, determine the first two 
eigenvalues and their corresponding eigenvectors. 

(b) For the case where kL4/EI = 5.0, determine the first two eigen- 
values and their corresponding eigenvectors. 


Prob. 10.8: The elastically restrained, uniform cross-section beam 
shown (Fig. 10.27) is subjected to a uniform lateral load of g lb per unit 
of length. The governing differential equation is 


dty _ kL* _ ql 
Bey) oe By 


where k is the spring constant of the foundation. The boundary condi- 


Figure 10.27 
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tions are specified to be 
y’”’(—0.5) = 0 y’”’(—0.5) = 0 
y(0) = 0 
y-EL.0) = 0g ie). 0) 50 


Using the collocation method (h = 0.25), determine the deflection pattern 
along the length of the beam, assuming kL1/EI = 230 and qL*/EI = 300. 
Prob. 10.9: Solve the following nonlinear initial-value problems for 
the range of independent variable indicated: 
CG ee GO NG oe ee 
fO) = 1.0; 0 <2.< 10 
f'0) =0;h = 0.05 
(b) § = uy — wy?) — yy; u = 0.95 
yO) = 14,0 <4£<1.0 
y/ (0) = 07h = 0.05 
(Note: For these problems, use the Kutta-Simpson one-third rule for the 
range 0 <t < 0.2, h = 0.05. Continue the solution by using the Milne 
method for 0.2 < ¢ < 1.0.) 


Prob. 10.10: For the nonlinear equation 
¢’ + ¢o¢ +2 =0 


the required boundary conditions are ¢(0) = 0 and ¢’(«) = 1.0; how- 
ever, the range of primary interest is only 0 < x < 2.0. 

[Note: In solving this problem, first assume that © can be considered to 
be 5.0. Next, obtain a second solution, assuming © corresponds to 10.0. 
If the two solutions thus obtained are (for the range of x of interest) 
within the accuracy desired, the values computed in that range can be 
considered to be the solution. If they are not, it is necessary to carry 
out a third solution assuming, for example, that © equals 20.0. With 
three or more independent solutions, obtained in the manner just described, 
extrapolation can be used to define the “true values.’’] 

Prob. 10.11: A 14WF43 pin-connected steel column of 280 in. length 
is subjected to a doubly eccentric axial thrust of 10,000 lb. The eccen- 
tricities are é, = 5 in. and ey = 5 in. If wu is the displacement of the 
shear center in the x direction, v is the displacement of the shear center 
in the y direction, and 8 is the angle of twist about the z axis, the equa- 
tions governing the deformation are 


vo’ = A(—5 —v + 58) 
u’ = B(-5 — u — 58) 
Be = Og: + D(—w’ + v’) 
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where A = 0.7/8 X 10-* 
B = 0.739 <X 10-5 
@ = 1.729 X< 10-4 


D = 0.856 X 10-6 
The boundary conditions are 
u(0) = v(0) = 60) = B”(0) = 0 
ua ol) — Bs L)r= 8 (L) = 0 


(It is to be noted that in all equations differentiation is with respect to 
the independent variable z.) Using the collocation method, assuming 
the total length is subdivided into four equal segments, determine uw, v, 
and 6 at the midpoint of the member (i.e., at 2 = 140 in.). 


numerical solution 
of partial 
differential 
equations 


11.1 Introduction 


For many problems the dependent variable is defined in 
terms of more than one independent variable. Variation 
with respect to the independent parameters is possible, and 
as pointed out in earlier chapters, the derivatives describing 
these independent variations are defined as partial deriva- 
tives. Differential equations involving partial derivatives 
are known as partial differential equations. 

Equation (10.3), which is here reproduced as (11.1), is a 
general second-order linear partial differential equation in 
the independent variables x and y and in the dependent 
variable z. 


0? 0 
Alea) 55 + Aslora) 5a + Aale) 53 + Ada) = 


If this sure is rewritten in the form 


022 
A(o,9) = 4: A2(x, De = at vas A3(z,y) mae 
dy 
Oz Oz 
a ae ye ;) = 0 C1123) 


318 
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‘ OZMOONG ; ; 
and if (.2,4),z, iat ay is allowed to be either linear or nonlinear, the 


partial differential equation (11.2) is classified as “quasi-linear.” 

In general, for the purpose of classification and study, second-order 
quasi-linear partial differential equations are grouped according to the 
nature of the conic associated with their characteristic quadratic form 


Q(n,e) = Ain? + Aone + Aes (11.3) 


The three possible cases and the necessary condition associated with each 
are defined in Eqs. (11.4). 


Elliptic equation: A? — 44143 <0 
Parabolic equation: A,? — 4A,A3 = 0 (11.4) 
Hyperbolic equation: A.2 — 44,A3 > 0 


Unlike ordinary differential equations, where for “real cases’’ a given 
equation may have either initial or boundary conditions associated with 
it, particular forms of second-order two-primary-variable partial differen- 
tial equations are almost without exception connected with particular 
types of associated conditions. Elliptic equations, for example, are of 
the equilibrium, or steady-state, type with boundary conditions of the 
Dirichlet form. That is, the boundary conditions require that either the 
function, or its normal derivative, or a linear combination of these must 
take on certain specified values on the closed boundary, within which 
boundary the solution is desired. On the other hand, parabolic- and 
hyperbolic-type equations are for propagation or non-steady-state situa- 
tions. For both of these types of equations, solutions are not restricted 
to a closed region but rather are allowed to propagate in an open domain. 
It must be understood, however, that this is not to say that for these 
types of problems conditions associated with certain of the variables can- 
not be of the boundary type. The distinction is that at least one is in 
an open domain. 

The torsional behavior of shafts, the potential distribution in a trans- 
former coil, and the potential flow of fluids are all examples of steady- 
state boundary-value-type engineering problems that require solution of 
the following elliptic partial differential equations: 


02d 02g 


V4 = + = 0 11.5 

Laplace’s equation{ oo) ie + 3y2 ( ) 
: 0p | do 

Poisson’s equation: V’¢ = —.+ — = f(x,y) (11.6) 
(aoe Oy’ 


+ V2 is pronounced ‘‘del squared” and is frequently referred to as the Laplace operator. 
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The one-dimensional heat-flow equation (11.7) is an example of a parabolic- 
type partial differential equation which is often encountered. 


Or ¥ OT 


ee! eee ce! ial se 
Ox? ot ( ) 


The one-dimensional wave equation (11.8), which can be used to define 
the vibration of a string, is a hyperbolic-type partial differential equation. 


ae Wa 


canes (11.8) 


It should be recognized that the problem types defined by Eqs. (11.4) 
depend on the magnitudes of A1, A2, and A;. These quantities, however, 
may be specified as functions of x and y. It is therefore possible for a 
given equation [of the (11.2) type] to be elliptic in one range of independ- 
ent variable, parabolic in a second range, and hyperbolic in still a third. 

While the discussion thus far has been restricted to second-order equa- 
tions of two independent variables, partial differential equations of other 
orders do exist, and have meaning. For example, the lateral deformation 
of an elastic plate is a steady-state (or equilibrium) type of problem, 
whose solution is governed by the biharmonic equationT 


04+ 0+ 0*d 
Vib = V2(V2) = 2 =F 
b= 1048) = teat ye Few) (119) 
The two-dimensional heat-flow equation is 
0°77 0?r Or 
=C : 
ae? ts ay? a (11.10) 


The ordinary finite-difference method, referred to in Chap. 10 as the collo- 
cation method, provides a simple and effective, yet general, method by which 
reasonable, quantitative solutions may be obtained at preselected pivotal 
points within the domain in question. The accuracy that is achieved with 
this method depends on the equation given and the grid type and size 
selected. 


11.2. Two-dimensional Grids 


Depending primarily on the shape of the boundary of the region for which 
solution is desired, the following types of two-dimensional grids are often 
used: 


1. Square 
2. Rectangular 


t V‘is pronounced ‘‘del fourth” and is frequently referred to as the biharmonic operator. 
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3. Triangular 

4. Hexagonal 

5. Parallelogram (‘‘skewed’’ coordinates) 
6. Circular 


These are illustrated in Fig. 11.1la through f, where it has been assumed 
that the independent variables are x and y and the dependent variable 
hyd CE 

In some cases, irregularly curved boundaries such as those illustrated in 
Fig. 11.2 are encountered. Boundaries such as these cannot be exactly 
represented by any of the grids shown in Fig. 11.1, and therefore special 


Figure 11.1 (a) Square; (b) rectangular; (c) triangular; (d) hex- 
agonal; (e) parallelogram (skewed coordinates); (f) circular. 


Z=F(x,y) Z=f(x,y) zZ=f(x,y) 


ZX, ZX) z=f(x,y) 
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Figure 11.2 


techniques are required. An example of one of the methods is discussed 
in Sec. 11.4. 

All the grid systems shown in Fig. 11.1 are directly associated with, 
and therefore directly applicable to, boundary-value situations. How- 
ever, as pointed out in Sec. 11.1, not all partial differential equation prob- 
lems are necessarily of this type. For the case where the independent 
variables are x and t, t being time, and the dependent variable is z(z,t), 
the situation probably would be more like that shown in Fig. 11.8. 
Here, the grid spacing in the x direction, the direction associated with 
the prescribed boundary conditions, is selected so that the total dimension 
is subdivided into an integral number of segments Ax. The greater the 
number of segments, the better will be the accuracy of the numerical 
approximation. In the ¢ direction, however, if the problem in that direc- 


Figure 11.3 
BEVAGGIE ir 
/ 
y 
2) ics en = 
p> ae 
l Do e a ae ie 5 
Mt =9——-9—_ 90 _G= = 
yl! 5 ———o o = = 
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5 C=) ay = 
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Z [Ax AxjAx idx] 
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tion is of the propagation type, there are no similar “geometrical con- 
straints” (such as division by an integral number). At is usually selected 
sufficiently small to ensure stability of the numerical process. This effect 
will be demonstrated later in this chapter. 


11.3 Development of Finite-difference 
Computational Molecules 


It will be recalled that the primary advantage realized in using the collo- 
cation method in the solution of ordinary differential equations was the 
ease with which the influence of the various terms in a given equation 
could be taken into account. The difference operators [for example, 
D*(y)] were initially treated as independent quantities, and their indi- 
vidual influences were cataloged. (See, for example, Table 9.27.) The 
composite effect of the various terms in a given differential equation was 
then taken into account by simple algebraic means using these pre- 
established relationships. This was possible because in each instance 
operations were written in terms of the function values at the preselected 
pivotal locations. This same procedure can be extended to the solution 
of partial differential equations. It should be recognized, however, that 
by the very nature of a multi-independent-variable situation as opposed 
to the one-independent-variable case considered before, cataloging will 
not be as convenient. 

To illustrate the general concept, it will be assumed that the system in 
question has two independent variables z and y. The dependent variable 
will be presupposed to be ¢(z,y). A rectangular grid of the type illus- 
trated in Fig. 11.4, where the spacing in the z direction is h and that in the 
y direction is k, has been selected. It should be noted that the coordinates 


Figure 11.4 
a 
H | ' (X9,Vo +h) 
eee een 8 o-Myg th) | or hyorh) 
| 0 
Soe ae | 
X0.o H k 
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Figure 11.5 


of any pivotal location can be defined in terms of a given reference loca- 
tion (%o,yo) by the expressions 


f= Lo + th Y = Yok jk (Lite) 


where 7 and 7 are integers. 

It will be recalled that in Chap. 10, in the solution of ordinary differ- 
ential equations by the collocation method, it was convenient for refer- 
ence purposes to number the various pivotal locations. The same is true 
for partial differential equations. In Fig. 11.5, five locations, centered 
symmetrically about location 0, are presumed to be of interest. By using 
the information given in Table 9.27 and assuming the acceptability of an 
error of the order of h? and expansion about the center location 0, the 
various ‘‘one-directional” partial derivatives are 


ag 


D,(¢) = On = Dh (—¢s + ¢1) (11.12) 
D.x(¢) = a = 7 (¢3 — 2¢0 + ¢1) (11.13) 
Dile) = 2 = (— es + 09 (1.14) 
Dylo) = 52 =F, (es — 2e0 + 09 (11.15) 


Ol iO cue Da CM ree ue RY ASR a a ee 


These equations can be represented by “stencils” or “computational 
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molecules” of the following type: 


CG) @) 
oS tt eal 
Dy ~ a AEE ep 
oS 


It should be noted that in each case the location about which expansion 
is to occur is shown in boldface. For example, in the first illustrated 
case, expansion is presumed about the center location. D, is then approx- 
imately equal to 1/2h times the quantity (—1 times the value of the func- 
tion at the first pivotal location to the left of the reference point, plus 1 
times the value of the function at the first pivotal location to the right). 
Note that such a computational-molecule representation of the operation 
d/dx is independent of the particular numbering of the reference locations. 

The finite-difference representation of the Laplace operator V’y is 
developed in the following fashion: 


Cie Op ad 1 
V9 = = —2 (oe 11.16 
g Ox? oe ay? he (¢3 go + $1) +: pe (¢4 go + 2) ( ) 


If the grid is presumed square, that is, = h, the finite-difference repre- 
sentation becomes 


1 
Vie SMa PO cia eh ath 200 ot ee) 


1 
= 2 (v1 + v2 + os + vs — 4¢0) (11.17) 


The corresponding computational molecule is 


(11.18) 


J 
nN 
2 
|= 
©) 
@ 
©) 
—— 
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The required relationship for the second mixed partial derivative 
d°y/dx dy, again assuming a square grid, is obtained by applying the 
operator D, to D,, or vice versa. In both cases the computational mole- 
cule shown in Eq. (11.19) is obtained. 


ee Dy y = (0) (0) (0) (11.19) 


A summary of the more often used square-grid computational molecules 
is given in Table 11.1. For those cases shown, it has been assumed that 
an error of the order of h? is permissible. The molecules were obtained 
by using the information contained in Table 9.27. 

Square-grid higher-order computational molecules for D,,, V’, and V4, 
assuming a permissible error of the order of h*, are shown in Table 11.2. 
Again, the information contained in Table 9.27 was used to develop the 
molecules. 

Two-dimensional integration formulas (i.e., volume determinations) 
can also be described as computational molecules. For the double 
integral 


Vi = lNif@y) dr ay Bt) 


Table 11.3 summarizes the trapezoidal and Simpson’s one-third rule 
cases. As was true in all the situations illustrated in Tables 11.1 and 
11.2, it has been presumed in Table 11.3 that the region in question can 
be subdivided into a square-grid system of dimensions h X h. 


11.3.1 ‘‘Skewed’’-coordinate Computational Molecules 


It should be evident that computational molecules can be developed also 
for noncartesian coordinate systems. When the spatial domains in ques- 
tion are parallelogrammatic in form, an often-used noncartesian system is 
the “skewed” one illustrated in Fig. 11.6. Here the coordinate direc- 
tions wu and v are at an angle a, one to the other. It should be noted that 
for the case illustrated, the coordinate dimensions u; and v;, which define 
the general point P;, are measured in the wu and v directions. In an zy 
cartesian system the corresponding situation is that shown in Fig. 11.7. 

If it is assumed for a given situation that the origins 0 of the two systems 
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Table 11.1 Computational Molecules (Square Grid): Symmetrical Par- 
tral Finite-difference Operators (Errors of the Order of h?) 


nt LOO} | = : s 

a ere (0) Dygy I —s © 

2 {OOO} SG Le 
ee @) w 
ee { O29-O)2) Gi) 2 C4) 
Daa oa {G4 +6)C4) OP a,,9 3 a) Cs) 
62) C4) 

LO) S) 


2 


ine) 
> 
wo 


shown in Figs. 11.6 and 11.7 coincide, that the directions u and x are one 
and the same, and that P,(u:,v;) and P;(z:,y:) define the same point, the 
two cases can be related as illustrated in Fig. 11.8. The transformation 


equations are 
z=u+tvecosea (11.21) 


y =vsina (11.22) 
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Table 11.2 Higher-order Computational Molecules (Square Grid): Sym- 
metrical Partial Finite-difference Operators (Errors of the 
Order of h*) 


(Because of the general nature of the process, the subscripts have not 
been included.) Independent variations of x and y with respect to wu and 
v result in the partial derivatives 


Ox Ox 
nae es 1 ee 
a “2 as COS @ 
(11.23) 
OUT Os ova 
ve ay 7 Sine 


If, for example, a function ¢ were given in terms of an xy coordinate 
system [that is, ¢(x,y)], Eqs. (11.21) and (11.22) and the relationships 
given by Eqs. (11.23) could be used to define the corresponding function 
g(u,v) and its partial derivatives with respect to these ‘new’ coordinate 
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Table 11.3 Computational Molecules (Square Grid): Determination of 
Volumes (Errors of the Order of h?) 


Trapezoidal rule 


pe 
[[fanax Lea 
A 


Simpsons one-third rule 


pe 
[[fanax ay > 
A 
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Figure 11.6 


directions. The partial derivatives would be 


0g dgdx Oy OY 0p (Xe) 0g 
win eee ee tele RL LE! RA 1 (0 ——— 11.24 
Ou Py cy Ox (r Wels ) Ox ( ) 
te) 0g 0 0p 0 0 ) : 

bi Sea + pe Pag (cos a) + —, (sin a) (11.25) 
Ov = 0x dv Ody Ov Ox Oy 
0? 0 fa 0? 
as ( “) —— (11.26) 
ou? = du \Ou Ox? 
a7 0 fo 0? a ‘ Oo . 
“eo (2 Se cosa = sin a cos a + — sin? a (ii 
dv? ~—s- dv \dv one Ox Oy Oy? 


dp ) (5°) dp pe. 
= = — cosa + sin @ 
du dv du \dv a Ox OY 


(11.28) 


In matrix form, the second partial derivatives with respect to the two 
different coordinate systems are related as in Eqs. (11.29). 


aes +1 0 0 
0’0 5 : 
5 =]cos*a sin?a 2sina cosa 
v 
0° 0 ; 
COs a sin a 
Ou Ov 


The inverted system is given by Eqs. (11.30). 


Figure 11.7 
VY 


020 
ax? 
020 
ay? 
020 


(11.29) 


Ox OY 
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Figure 11.8 
070 02 
ae +1 0 0 — 
Ox Ou? 
020 cos? a 1 2 cosa 02— 
oe. = tet ; = : 
Oy? sin? a ‘5 sin? a sin? a ov? 30) 
0’ cos a 1 079 
aa 0 : 
Ox Oy sin a sin a Ou Ov 


The laplacian defined in the zy system can now be expressed in terms 
of the ‘‘skewed”’ wv coordinate directions. 


079 = 09 ~d20 1 ( oe 070 
Vo = — + = - cos? a — == = 2° 
i 2 sin? a i Ou? T Ov? re Ou Ov 


1 020 020 070 
or V4o = - = = 72 = iL Bil 
é sin? a (= ee du Ov 5 Ov? ca 


Note that if a = 90° (that is, the coordinate system is a cartesian one), 
Eq. (11.31) reduces to 


ao dy 


By using Eq. (11.31) a computational molecule whose error is of the 
order of h? can be developed. Again, for the initial definition, pivotal 
locations are referenced by numbers. The particular ones and the 
arrangement selected are shown in Fig. 11.9. In this development, at 


Figure 11.9 
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this stage, it will not be assumed that the interval h is equal necessarily 
to &. Using the information contained in Table 9.27, 


0p 1 
Pies Gat (ys — 2¢0 + ¢2) (11.32) 
070 1 
= = Lies 
Bir Oe Ahh (¢4 96 2 55 a) ( ) 
020 il 
av? = p2 (gs — 2¢0 + ¢1) (11.34) 


Substituting these values into Eq. (11.31) yields 


1 1 2cosa 
ENE, E ima gem aioe 0 arti is 2) 
1 
+} Bp (¢5 — 2¢0 + e) | (11.35) 


If there is introduced a nondimensional parameter y = £/h, Eq. (11.35) 
can be written as 


il 
py 2h?y? sin? a {gol —4(1 + y?)] + ¢1(+2) + ¢2(y cos a) + ¢3(277) 
+ pa(—7 cos a) + ¢3(+2) + ¢o(¥ cos a) 
+ o7(2y7) + ¢3(—y cos a)} (11.36) 


The computational-molecule representation is then that shown in Fig. 
11.10. Assuming h = &, Table 11.4 summarizes ‘‘molecules” for V2¢ 


for angles of a equal to 30, 45, 60, and 90°. 


Figure 11.10 
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Table 11.4 Computational M olecules [‘“Skeweqd” Grid (h = ft = )): Symmet- 


rical Partial Finite-difference Operators for Vo (Errors of the 
Order of h?) 
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11.3.2 Triangular-coordinate Molecules 


Another noncartesian coordinate system frequently used consists of a 
triangular lattice of the type shown in Fig. 11.11. 

Using a procedure similar to that employed in See. 11.3. 1, the two 
coordinate systems can be related, as shown in Bree 11,12; 


t=u+veosa+weosB (137) 
vsina + wsin B (11.38) 


ll 


y 
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Figure 11.11 


The partial derivatives of x and y with respect to 
as Eqs. (11.39) and (11.40). 


Ox Ox Ox 
eta — = cosa — = 
Ou ov Ow 
oy dy oy 
— = 0 — =sina — = 
Ou Ov Ow 


u, v, and w are listed 


cos B 


sin 8 


Using the same procedure as for the ‘‘skewed”’ case, 


ep dy 

uz ax? 

320 020 BY 

2 = 5x2 cos? a + 2 

de #8 ag po 2? 

Ow? Ox? Ox Oy 
Figure 11.12 
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(11.39) 


(11.40) 


(11.41) 


(11.42) 


(11.43) 
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This gives for the laplacian 
az 0 

Wes gd aga 

Ox* ~— dy? 


1 eo . 070 
= 2 —_ ee SL 9) 
2 sin a sin 8 sin (6 — a) le ag Ov? ee 


2 


a 
= sit 2a | (11.44) 
Ow? 


For an equilateral triangular-coordinate system, where a = 60° and 
B = 120°, Eq. (11.44) reduces to 


sag 020 070 020 
ns be rs Oe 


Computational molecules for this and the corresponding biharmonic 
operator are given in Table 11.5. 


11.3.3 Polar Coordinates 


In polar coordinates, where the point P; is defined in terms of a distance 
p and an angle @, the transformation equations are obtained from a 
consideration of Fig. 11.13. 

x = pcos #6 (11.46) 


y =p sin 0 (11.47) 


By using the process hereinbefore described for both the ‘‘skewed” and 
the ‘triangular grid” case, it can be demonstrated that the laplacian 
in polar coordinates is given by 


Wig ta (11.48) 

With the notation of Fig. 11.14, the computational molecule for the 
polar coordinates is that given by Fig. 11.15a, where y = &/h.  Fre- 
quently, these computational molecules are also shown in the form given 
in Fig. 11.150. 


11.4 Irregular Boundaries 


For most equilibrium-type problems the value of the dependent function 
is specified along the boundary. In the torsional analysis of a shaft, for 
example, where it is required that V?g = C within the interior of the 
region in question, it is necessary that the function ¢ have a certain 
specified constant value on the boundary. 
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Table 11.6 Computational Molecules (Equilateral Triangular Grid): Sym- 
metrical Partial Finite-difference Operators 


Error of the order of A®@ 
> 
Vv 
= BQ \ 
eee 
2 
Wes = = = ° 
Keo eine 
u No 


Error of the order of A* 


eh weal ie t 
Vee peosoes 


* oe to—f-L2) 
nie | QAR 49-24 
eat Ono eS 


SS 


In Fig. 11.2 it was illustrated that not all boundaries necessarily 
coincide with pivotal locations associated with regular grids. While 
there are available several different methods for handling such situations, 
the one that will be here illustrated presumes that it is desirable to develop 
a computational molecule which makes use of the actual boundary 
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Figure 11.13 


points, rather than those which fall ‘‘outside” and are associated with the 
continued regular grid. The cartesian-coordinate case is illustrated in 
Fig. 11.16. Points ©, @, and @) fall within the region in question, 
whereas pivotal locations @) and @ lie outside. (The assumed boundary 
has been shown crosshatched.) The problem is to develop the required 
computational molecule for location ©, presuming a nonsymmetrical 
grid pattern defined by locations ©, @, @, @, and @. Such patterns 
are often referred to as unequal-leg stars or unequal-leg star points. 

If it can be presumed that the function in question, ¢(z,y), can be 
represented to a sufficient degree of accuracy by a second-order expression 
in each of the coordinate directions, then it can be shown that 


0 1 
= |=; as eh eal, Mareen (11.49) 


1 
Ae alt ee a(l + a) | 


oes |=; eke 1a sl ee 11.50 
pre eemieeot  mogeces el ia)” Cele 
020 2 $1 Yo Ya 

dp _ 2 a 0 thy Oe ae: 151 
Ox? =| a + | ( ) 
es =| DD a es (11.52) 
oy? A? LIit+ 6 B( 1-6) 


Figure 11.14 
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Figure 11.15 


Figure 11.16 
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|= 
Bh 

Cae 

h 

ve 
Figure 11.17 

The laplacian is 
2 $1 $2 ~o 0 fa $b 

Dine aoe oe = Le 
- bearers, 5) aecs) See) oe 


The associated computational molecule is shown in Fig. 11.17. For the 
case where three of the “‘points” of the “‘star’’ are located away from the 
© location at distances less than h, the computational molecule is given 
by Fig. 11.18. 

As an example of how these molecules would be used, consider the 
specific case defined in Fig. 11.19. With the information contained in 
Fig. 11.17, the value of the laplacian at location ©, in terms of the 
function values at points @, @, @, and ©, is that given in Eq. (11.54). 

1 1 1 
o(ate ag) tersa3t mises 


1 1 
T #08408) 060 + | 


Figure 11.18 
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Figure 11.19 


2 
or V’¥9 = ip (—2.92¢) + 0.56¢1 + 0.63¢2 + 0.69¢. + 1.04¢,) (11.54) 


For those situations where on the boundary the value of the normal 
derivative is specified, rather than the value of the function itself (as was 
presumed in the preceding cases), a somewhat different approach is 
often used. For illustration of the method, consider the particular situa- 
tion defined by Fig. 11.20. It will be assumed that the point of expan- 
sion is location ©). Adjacent points @, @, and @) are presumed to lie 
within the domain in question, whereas point @) falls without. It 
should be recognized that the direction of the normal to the boundary 
does not coincide with the assumed grid directions. 

Figure 11.21 is an enlargement of a part of Fig. 11.20. If there is 
constructed through point @ a perpendicular to the boundary (inter- 
section with the boundary is presumed at point @), it is possible to define 
the function value at location @ in terms of the function values at © 
and () and the value of the normal derivative at @. 


Figure 11.20 
© @ 
s\ v i 
h @ © Boundary 
{ ° 
Lo) ee 
V = ne = 
h h 
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Figure 11.21 


By referring to Fig. 11.21 and assuming linear interpolation, the value 
of the function at some location P along a line joining the pivotal points 
©) and @) can be expressed as 


gp = 05 + (go — ¢s)(1 — a) (11.55) 
or gp = a(gs) + (1 — a)go (11.56) 
If linear interpolation is again assumed, the value of the normal deriva- 
tive at location @ is 
(e) = 
(5*) _ €4 — oP (11.57) 
On 6) Lo 
0 
or OP = "04 aa Lo ce (11.58) 


Eliminating gp from Eqs. (11.56) and (11.58) yields an expression for 
ga in terms of go, gs, and (d¢/dn)@. 


F) 
gs = (1 — a)go + ags + fe ek (11.59) 


Equation (11.59) can be used directly in the establishment of modified 


finite-difference expressions. For example, the laplacian 


1 
Vo = he (y1 + v2 + 93s + os — 4¢0) (11.60) 


1 F) 
becomes V’¢o = 2 {es + g2 + ¢3 + ags + ty a). 


—{(4-(- «)el} (11.61) 
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Figure 11.22 


When two points fall outside the domain, a situation of the type 
defined in Fig. 11.22 must be considered. Using the same procedure as 
in the previous case yields the following: 


fe) 

vs = (1 — y)¢o + veo + fs (=*) (11.62) 
n2/C) 
dy 

eo 1 Bee + Bon + (22). (11.63) 
Ont 


The lente then becomes 


Vo =Fa{ + Be + 1+ ver + (2 =) ae es 2) 
Om Ae PA) 


— 4-1-6) - = wool 64) 


When the point of expansion of a “star’’ is located on the boundary, 
and it is required that the normal derivative at that point take on a 
particular value, a still different situation must be examined (see Fig. 
11.23). For this case, the.laplacian becomes 


i 
2 
ein h? eo i epee ae “ (ale ¢ tar 


~4--s- C= ole} (168) 
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It should be recognized that when compared with points remote from 
the boundaries, Eqs. (11.61), (11.64), and (11.65) are less accurate. 
While it is possible to develop other, more accurate methods for handling 
the normal-derivative boundary-value problem, these other methods 
will not be considered in this introductory text. 


11.5 Elliptic Partial Differential Equations: 
Boundary-value Problems 


Example 11.1 As a first example of the numerical solution of two- 
dimensional elliptic boundary-value-type partial differential equations, 
consider the problem defined in Fig. 11.24. The region of interest is 
square and has a side dimension of a. The specified “edge loadings” are: 


Along x = 0: e=10-" 
Along x = a: g=0 
(11.66) 
Along y = 0: e=10-— 
Along y = a: g =0 


Within the given boundaries, it is desired to determine the distribution 
of ¢ such that 
V7o_= 0 (11.67) 


It will first be assumed that a three-equal-part division of each of the 
coordinate dimensions (that is, h = a/3) will result in a sufficiently 
accurate solution. The chosen grid system is shown in Fig. 11.25. 


Figure 11.23 
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Figure 11.24 


Four unknown values of ¢, at pivotal points @, @, @, and @, are 
presumed. At each of these locations, it is required that the laplacian 
(in finite-difference form) equal zero. 

From Table 11.1, the computational molecule associated with V? is 


ee 4)}—€4}(1) (11.68) 
Gc 


Since Eq. (11.67) requires that V?¢ = 0, it is sufficient to require at each 
of the interior points of expansion that the bracketed term in Eq. (11.68) 
equal zero. Consider for illustration expansion about location @, as 
shown in Fig. 11.26. The unknown values of the functions ¢ at each 


Figure 11.25 
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Figure 11.26 


location are also noted in the figure. The corresponding equation is 
(+1)(4) + (41)(¢2) + (4+1)(y2) + (41)) + (—4)(¢1) = 0 (11.69) 


Similarly, equations are written for each of the other interior points. 
The resulting set of equations is 


ee 1 0 Y1 —-kk 

ie 0 $2 0 
= (11.70) 

1 0 44 Y3 —48 

o 1 bt =4. JLo) L-% 


The solution, which can be obtained by any of the methods defined in 
Chap. 5, is 


g1 = 0.22 ge = 0.11 gs = 0.44 ones ier al) 
This solution is illustrated in Fig. 11.27. 


Figure 11.27 
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Example 11.2 As a second example, the Poissonian equation 
Veo +2=0 


is prescribed for the interior of the region defined in Fig. 11.28. On the 
boundary it is required that ¢ = 0. (The selected grid system has been 
indicated on the figure.) It should be noted that points @, @, and @) 
require the development of unequal-leg stars for the Laplace operator. 
With the information contained in Fig. 11.17, these are found to be: 


Location @): 
72 wt, (11°72) 

Location @): 
(11.73) 

Location @): 
(11.74) 


The equations governing the solution are listed as Eqs. (11.75). 
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Figure 11.28 
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CE1E75) 
The solution is 
g1.= O.5dih? ¢s = 1.488h? gg = 1.300h? 
Ge LASIh? gs = 2.101h? gio = L712h2 
(11.76) 
¢g3 = 1.483h? ¢7 = 2.074h? gu = 1.446h? 
Oo. = 1 187h2 gs = 1.315h? 
which is illustrated in Fig. 11.29. 
Figure 11.29 
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Figure 11.30 


Example 11.3 As an example of a higher-order boundary-value-type 
partial differential equation problem, consider the solution of the follow- 
ing biharmonic equation: 


049 d4Q Op 


PORE en Ube v= (ig 


The domain in question is shown in Fig. 11.30, as are the prescribed 
boundary conditions and the selected grid system. Three unknown 
pivotal values have been assumed: 91, g2, and g3. To facilitate the 
handling of the specified boundary conditions, imaginary pivotal loca- 
tions outside the domain will be presumed. Figure 11.31 shows these 
imaginary locations and their corresponding ¢ values—related to the 


Figure 11.31 
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interior points—as well as the general form of the computational molecule 
for V‘ placed “over” location @ for evaluation. From Table 11.1, the 
computational molecule to be used for V? is 


(11.78) 
The controlling equations are therefore 
20. —8 —8 1] ¢1 hk 
—8 “19. 1 || e| =| aK (11.79) 
She Mie EAN os htK 


It is to be noted that the special equations for handling normal- 
derivative boundary conditions that were derived in Sec. 11.4 were not 
used in this example. This was due to the coincidence of the assumed 
grid pattern and the normals to the boundaries, and defined zero values 
of the function on these boundaries. 


Example 11.4 To illustrate the use of the previously developed 
polar-coordinate computational molecule (Fig. 11.15), consider the prob- 
lem of finding the distribution of a function ¢ over a cross section of a 
keyed, hollow, circular shaft, when ¢ is governed by the equation 


Vo =K (11.80) 


It is required that ¢ = 0 on the boundary. The specific case to be 
examined is illustrated in Fig. 11.32. It is to be recognized that sym- 
metry exists, and therefore only one-half of the cross section need be 
considered in any numerical computations. 

The concentric-circle-and-radial-line polar-grid pattern shown in Fig. 


Figure 11.32 
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Figure 11.33 


11.33 will be used. For reference purposes the rings have been labeled 
with lowercase letters; whereas the radial lines have been numbered. 
Pivotal location has been noted on the sketch. The entire 360° 
circle has been subdivided into 16 equal parts (A@ = 7/8 rad). For 
ring b, the p distance is }4r._ For ring c, the radius is 34r. 

With the information shown in Fig. 11.15, the computational molecules 
associated with rings b and c are shown as Eqs. (11.81) and (11.82). 


Ring b: 
gwd G6) Gist (11.81) 
Ring c: 


(11.82) 


At location , the expanded equation is 


Kr? 
11.5 


The entire set of governing equations is given as Eqs. (11.84). 


A 78opeeOO ce al 00 pea (11.83) 
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The solution is 
gw = —0.0362Kr? gx» = —0.0612Kr? 
gs = —0.0669Kr? on = —0.0679Kr?* 
gn» = —0.0394Kr? gsc = —0.0540Kr? 
0. 0580K7 ane 
gx» = —0.0530Kr? ga = —0.0655Kr? 
ge = —0.0676Kr? os, = —0.0680Kr? 
go = —0.0424Kr? gic = —0.0576Kr? 
¢goc = —0.0594Kr? vg, 


—0.0596Kr? 
(11.85) 


—0.0596 Kr? 


Example 11.5 The buckling of a plate is a boundary-value eigen- 
value-type problem that can be conveniently solved by using the finite- ° 
difference methods heretofore developed. For illustration, consider the 
particular case shown in Fig. 11.34. The plate, which is square, is 
presumed to be of uniform thickness. It is “‘fixed’””—that is, dw/dy = 0 
and w = 0—along one of its edges, but “‘simply supported” —0?w/dx? = 0 
and w = 0—along the other three edges. A force per unit length of 
N, is assumed along two opposite edges, as shown. ‘The problem is to 
determine the critical value(s) of Nz such that the plate can deflect 
(buckle) out of its original plane. 

The governing fourth-order partial differential equation is 


Wipe ee 0 11.86) 
dx? oe 


where w = lateral deflection of plate 


oN 
= 5 (11.87) 


D = plate bending stiffness 


Figure 11.34 
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From Table 11.1 the necessary computational molecules are 


(11.88) 


and 


(11.89) 


Equation (11.86) can therefore be expressed in finite-difference computa- 
tional-molecule form. 


Ee) (11.90) 


If it is assumed that a sufficiently accurate solution can be obtained 
by dividing each of the sides of the plate into four equal parts, the grid 
system to be investigated is that shown in Fig. 11.35. It should be noted 
that imaginary pivotal points outside the plate also have been shown. 
By using the specified boundary conditions, these are related to the 
unknown interior values. 

Applying Eq. (11.90) at each of the interior pivotal locations gives 
the set of equations (11.91) (on the following page): 
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Figure 11.35 


For this set, the first five eigenvalues are: 


Ah? = 3.032 or AL? = 48.512 


= 3.563 = 57.008 
= 4.820 = 77.120 (11.92) 
= 8.504 = 136.064 
= 8.880 = 142.080 


Example 11.6 As a second eigenvalue example, consider the deter- 
mination of the natural frequency of vibration of a skewed, simply 
supported membrane. The dimensions of the membrane to be examined 
are given in Fig. 11.36. The governing partial differential equation is 


V-w + rdw = 0 (11.93) 


Figure 11.36 
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where w = lateral deflection of membrane 
Ar mL? 
= 96 
= mass per unit of area 
tensile force per unit of length of membrane 
= natural frequency 
The “composite” finite-difference computational molecule associated 


with Eq. (11.93) is 


2 


8 tas 
| 


p =O (11.94) 


EY 


n = L/h = number of equal-length segments into which each of the 
sides is divided. 

First, assume that a sufficiently accurate solution can be obtained by 
dividing each of the sides into two equal parts; that is, n = 2. This 
case is illustrated in Fig. 11.37. Since it is specified that w = 0 along 
the boundaries of the membrane, the application of Eq. (11.94) to the 
case defined by Fig. 11.37 (about location @) yields 


2) sin? 
(-s nm =) wi = 0 (11.95) 
16 
= : 
zs sin’ a aL) 


Secondly, had n been selected equal to 3, the situation would be that 


Figure 11.37 
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Figure 11.38 


illustrated in Fig. 11.38. Applying Eq. (11.94) to each of the indicated 
pivotal locations results in the following set of equations: 


“S 2d sin? a 
-8+ =. 2 2 cos a W1 0 
2d sin? a 
Zz —-8+ =e —cosa 2 we 0 
2d sin? a 
2 —cos a —8-+ oe 2 W3 0 
Ss 2d sin? a 
cos @ Z 2 -—8+ ; w4 0 
SS 


(11.97) 


Thirdly, Fig. 11.39 is for that case where n = 4. The controlling 
equations are those listed as Eqs. (11.98). 


Figure 11.39 
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For the particular two cases where a = 60° and a = 90°, the smallest 
eigenvalues [obtained from Eqs. (11.96) to (11.98)] are given in Table 
11.6. Also listed are \ values obtained by extrapolation to that limiting 
case where 1/n? = 0. 


Table 11.6 
Number of Divisions a = 60° a = 90° 
To Eq. (11.96) NS MI.38} > = 16.00 
a— Kgs. (11.97) = BRS > = 18.00 
n=4 Eqs. (11.98) IN Ss DALY? ON = USES 
Extrapolation to the case » = 25.29 » = 19.74 
where n = 


Extrapolation was carried out by using the Lagrange formula (9.73) 
applied to the case illustrated in Fig. 11.40. It is assumed that there 
are three known values: %1,A1; %2,A2; and %3,A3._ The desired equation is 
(xo — %2)(Xo — ts) (to — %1)(%o — Zs) 


Ai + 


Xo 


2 


s (cy ito) (tr — Ls) (S2-—==71) (He. —"2s) 
ae en 2) i199) 
(zz — 21)(%3 — 42) 
or No = 3.04761 — 2.3148), + 0.2667); (11.100) 


11.6 The Use of Relaxation Methods in the 
Solution of Boundary-value Problems 


11.6.1 Unit Relaxation 


As pointed out in Sec. 11.5, since equilibrium-type problems—solved 
by finite-difference methods—result in the solution of simultaneous 
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linear algebraic equations, any of the methods defined in Chap. 5 can be 
used. Relaxation methods (Sec. 5.5) are often encountered. 


Example 11.la Consider again, as a first demonstration of the 
method in question, Example 11.1. In relaxation form, the equations 
to be solved [Eqs. (11.70)] are 


—49,+ g2t ¢2 +43=fi 

gi — 4¢2 + ¢ = Rs 
(11.101) 

71 — 493 + git 44 =; 


The unit-relaxation table is listed in Table 11.7. The relaxation solution 
is given in Table 11.8. 


Table 11.7 


Agi Age Ags Ags AR, AR2 AR; AR, 


Table 11.8 
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The solution is therefore 
g, = 0.22 g. = 0.11 v3 = 0.44 gs = 0.22 (11.102) 


An alternative procedure for carrying out this relaxation is frequently 
used for boundary-value-type partial-differential-equation problems. 
Actually, the relaxation procedure is the same, but the “bookkeeping” 
is handled in a somewhat different fashion. 

It is to be recognized at the outset that the unit-relaxation table 
listed as Table 11.7 can be deduced directly from a consideration of the 
finite-difference computational-molecule representation of the Laplace 
operator. 

CG) 


Oe Or (11.103) 
CG) 


Consider, for example, the influence of a unit change in the value of 
¢1 on the definition of the ‘‘equilibrium equations” for the four place- 
ments (see Fig. 11.41) of the “molecule” given as Eq. (11.103). For 
each of the first three placements, i.e., cases a, 6, and c, the unit change 
in ¢ at location @) results in a change in the value of the laplacian, and 
therefore in the value of the residual. For case d, however, no influence 
is realized. 


Figure 11.41 (a) Eq. (1); (b) Eq. (2); (c) Eq. (3); (d) Eq. (4). 
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It should further be recognized from a consideration of the molecule 
for the laplacian [Eq. (11.103)] that a unit change in the ¢ value for 
location 7 results in a change in R; of —4, and a change of +1 in each of 
the adjacent R’s. This, of course, does not hold for the boundaries, where 
normally ¢g’s are specified, and are therefore not subject to relaxation. 

Since the influence of altering any one variable is restricted to a small 
region, it is comparatively easy to keep a “running account” of values of 
g and R directly on a grid which represents the domain in question. For 
example, the arrangement shown in Fig. 11.42 is frequently used. 


Example 11.16 For the example problem just considered, the 
“known” or given ¢ values are those listed in Fig. 11.43. By multi- 
plying each of these ¢ values by 100 and selecting as “initial guesses’’ 
values (for 100¢) at the four unknown interior pivotal locations, possible 
starting values for relaxation are those given in Fig. 11.44. Note that 
the largest residuals (—57) occur where 100¢ was assumed to be 40. 
A first relaxation at either one (or both) of these locations therefore 
should be undertaken. Since a change of Ag; will alter R; by a factor 
of —4(A¢;), an addition of —14 to the +40 will reduce the residual 
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(—57) to —1. This is shown in Fig. 11.45, where corresponding changes 
in all the influenced residuals are indicated. It should be observed that 
there was no “carryover” to the external points. 

Figure 11.46 is a complete “unit-relaxation solution,” carried out in 
the manner just described. Convergence to the true solution is fairly 
rapid. It should be recognized, however, that for the case illustrated, 
the alternating + and — residuals tend to cancel each other. If the 
net sum of the residuals in an area is not equal to zero, these cancella- 
tions will not take place, and the solution will converge only after the 
residuals are ‘‘moved” by relaxation to the boundaries. This can be 
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a lengthy process, since residuals change back and forth in sign but 
change little in magnitude. To circumvent this difficulty, group relaxa- 
tion is often used. 


11.6.2 Group Relaxation 


As demonstrated in Example 11.1), if ¢ at a given pivotal location is 
changed by a value of +1, the value of the residual at that location is 
altered by —4, whereas the values of the residuals at each of the adjacent 
pivotal locations are changed by +1. When ¢ values at several different, 
adjacent locations are changed, the relaxation pattern is altered. Con- 
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(a) P 


Figure 11.47 (a) Regionin question; (b) combined Laplace oper- 
ator; (c) resulting group operator. 


sider, for example, the case illustrated in Fig. 11.47, where two adjacent 
points are altered simultaneously by a value of +1. The group-relaxa- 
tion operator illustrated in Fig. 11.47c is frequently illustrated in the 
fashion of Fig. 11.48. Had three consecutive pivotal locations along a 
given grid line been subjected to changes in ¢ of 1, the group-relaxation 
operator would be that shown in Fig. 11.49. Figure 11.50 represents 
still another group-relaxation operator. Note especially the pattern at 
the reentrant point. It is also to be observed that the residual at the 
centermost point, where there are no grid connection lines to the 
“outside,” is not changed during the group-relaxation process. 

In comparing Figs. 11.48 to 11.50, certain common features are 
recognized. First, at each interior pivotal location—that is, interior to 
the region of group relaxation, where it is presumed that all y values are 
altered by a value of +1—there will be a change in the residual at that 
point equal to —1 times the number of direct connections of that location 
to an exterior non-altered pivotal location. Secondly, at each “‘station- 
ary” external location, there will be a change in the value of the residual 
equal to +1 times the number of direct connections to interior points. 


Example 11.1lc Example 11.1b could be group-relaxed as follows. 


The presumed starting values are those given in Fig. 11.51. If a relaxa- 
tion that involves all the interior pivotal locations is assumed, the group 
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operator shown in Fig. 11.52 is obtained. It is to be remembered that 
any residual change moving into a boundary will simply be absorbed. 
There will be no change in the boundary value. 

For the case in question (Fig. 11.51) the net residual within the 
selected interior region is 


2h = —o-— 5i— 0+ 14.= —100 (11.104) 


It is desirable that this be removed—one part along each of the eight 
externally connecting paths. By selecting a change in ¢ at each interior 
pivotal location equal to —12, the corresponding change in RF (at each 
of these interior locations) will be 


= (—2)(—12) = +24 (11.105) 


The total change within the region is (4)(+24) = +96. An initial 
group relaxation of the type just described results in the values shown 
in Fig. 11.53. From this stage, relaxation would normally proceed in 
the unit-relaxation fashion described in Example 11.10. 


Example 11.7 It is specified that within a rectangular region 
d X 1d the laplacian 


g g 

Vo = —— + — = 0 11.106 

OF ar oy? ( ) 

must be satisfied. Moreover, the required boundary values are those 
listed in Fig. 11.54. It is to be noted that along part of the boundary 
y is given, whereas, on the remainder, it is required that the normal 


derivative of ¢ equal zero. 
The first step in the numerical solution of this problem is the selection 
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of a suitable grid. The chosen one is shown in Fig. 11.55. (Also 
included in this diagram are the imaginary pivotal locations required for 
the satisfaction of the specified boundary conditions.) The governing 
relaxation equations for the determination of the 19 unknown ¢ values 
are as follows: 


At location 2a || ¢ + 100 + ox + 50 — 4¢o = Roa 
2b || Goa + 100 + go + Gr — 462 = Row 
2¢ || go + 100 + goa + Ge — 4¢2 = Roe 
24 || G2 + 100 + gee + Ysa — 400 = Rea 
Ze 2d + 100 ak Por == P3e — 4 Gr = Roe 
2s G2. 4+ 100 = 2g + P3f = A gos = Roy 
29 Por a 100 a. 100 — 3g — 4Q0 = Roy 
35 || 50 + gx + gsc + ve — 4¢2% = Rap 
de 3b + Pr “ta 3d ot P4eo — 4G 3¢ rE Rc 
3a || Pc + gra + se + gra — 4924 = Roa (11.107) 
de P3d == Pre = P3f == Pte — 4036 ae Rs. 
37 || Gse + poy + Gag + gar — 437 = Ray 
39 || Gar + Gag + 100 + vag — 4029 = Rag 
4, || 50 + gs + se + 92 — 40m = Ra 
4, P 4b +a P3c ai Pad ae P3c — 4 4¢ = lige 
4a|| gsc t+ ¢sa + pae + Ysa — 44a = Raa 
4. || psa + Gae + gay + Ose — 4040 = Rae 
4; || pre + gar + Gag + G35 — Soar = Ray 
4, || pas + 93g + 100 + vs — 44g = Ray 


(It is to be understood that, while these equations have been written in 
standard relaxation form, the set can be solved by any of the methods 
defined in Chap. 5.) 
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As a first approximation to the values of ¢ at the interior pivotal 
locations, the ones given in Fig. 11.56 will be presumed. The correspond- 
ing residuals are also listed. These estimates can be improved and con- 
vergence of the numerical process speeded up by going through a group 
relaxation for a part of the total region in question. The part to be 
examined is shown in Fig. 11.57. The group-relaxation operator is 
given in Fig. 11.58. From Fig. 11.57 it is observed that the net sum of 
the residuals within the region in question is 


Zk = 10 + 10 +.10 + 10 + 30 + 10 = 80 (11.108) 


This must be dissipated in equal amounts along the 12 externally connect- 
ing lines. Therefore 


Titeange along each external line — 8U > = ef (11.109) 
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Figure 11.58 Group-relaxation operator. 
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There results the modified form for Fig. 11.56 shown in Fig. 11.59. 
The unit-relaxation table for the total problem is given as Table 11.9. 
The final values of ¢ and R obtained by unit relaxation are given in Fig. 
11.60. Contour lines of the electrical analog solutions to this problem 
are also indicated for purposes of comparison. 

The relatively large errors that exist between the obtained numerical 
and the analog solutions are due almost entirely to the size of the selected 
grid in areas where ¢ must change rapidly. This effect can be corrected 
by completely reworking the problem using a finer mesh, or—as will 
be demonstrated—reworking the problem in the ‘“‘troubled area’’ only, 
using the finer mesh and starting from the values obtained by using a 
coarse net. 


11.6.3 Change in Grid Size 


Example 11.7 illustrates the fact that even for one given problem, 
depending on the boundary conditions imposed, various degrees of 
accuracy may be desired. For a uniform spacing, the required accuracy 
in the most critical region dictates the entire pattern, and there results 
a large increase in the number of equations to be solved. A better 
procedure would be to use a varying grid size—one which allows a finer 
mesh in regions of rapid change, and coarser spacing where the variation 
is gradual. 

To facilitate solution, if adjacent regions of unequal grid size are 
presumed, there must be available finite-difference operators which 
bridge the gap between the coarse and fine nets. If patterns having 
spatial dimensions related by factors of 2 are presumed, the possible 
types of interfaces are those illustrated in Fig. 11.61. Five points within 
the region are noted. These result in the five special spatial configura- 
tions shown in Fig. 11.62. It is to be recognized that equal-leg cases 
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Figure 11.61 


a and e, that is, configurations @) and @), are of the same type, the grid 
size being the only difference. Points @) and @ represent unequal-leg 
stars whose solutions to these are available in Sec. 11.4 (Fig. 11.17). 
Points of the configuration @) type, i.e., case c, however, require special 
attention. 

To establish the molecule associated with case c, the coordinate system 


Figure 11.62 
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shown in Fig. 11.63 is assumed. It is further defined at the outset that 
variations in ¢ within the region in question (114h X h) can be represented 
to a sufficient degree of accuracy by the polynomial 


eg = ax + by + cry + dx? + ey? + oo (11.110) 


The coefficients of this equation can be written in terms of the function 
values at pivotal locations 1 through 5 as follows: 


“h Yh —3eh? he Mh ][fa] [oo] [or 
doh 14h? || b 0 ~2 

Loh 4h? c|+]| ¢o| =] ¢s CURT) 
— wh 4h? || d 0 $4 

—h —Wh Ye Matte} Lol Les 


Solution of this set of simultaneous equations gives, for example, 


1 

d= 3h2 (—4¢0 + ¢1 — 92 + 4¢3 — gs + ¢s) (11.112) 
D 

e= he (g2 = 200 = ¢4) Chek) 


If the computational molecule for the laplacian is desired, that is, 
Vig (11.114) 


the necessary derivatives of Eq. (11.110) are 


ae dy 
— = 2d == S72 11.115 
ox oy? ‘ ( ) 


Substituting these into Eq. (11.114), and in turn Eqs. (11.112) and 
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(11.113) into the resulting expression, yields 
1 
Vo = Bh? (—32¢0 + 291 + 10¢2 + 8¢3 + 10g, + 2¢5) (11.116) 


Table 11.10 is a summary of the molecules required for solution of prob- 
lems whose assumed unequal grid sizes have spatial dimensions related 
by a factor of 2. 


Table 11.10 Laplacian Computational Molecules 
(Unequal Grids, Error of the Order of h?) 
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With regard to the general question of what relative proportions of the 
total region in question should be subdivided, no clear rules exist. One 
could, for example, presume only a “unit”? width of subdivided region 
in the area of anticipated greatest rate of change. For the example 
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just considered, such an assumption would not result in any major 
computational difficulties, and the entire region could be considered in 
solving the problem. Then again, possibly two widths (or lengths) 
should have been presumed, as illustrated in Fig. 11.64. The resulting 
39 unknowns could undoubtedly be handled. However, for other cases 
where larger regions are considered, it may be desirable to presume that 
the function values obtained from the coarser-grid analysis are constants 
at distances of three, four, or five grid spacings from the boundaries of 
the subdivided region. 


Example 11.8 ‘To illustrate the procedure for mixed-grid problems, 
consider the region defined in Fig. 11.65. The rows have been numbered 
@ through @ and the columns @ through @). Interior point types 
corresponding to those illustrated in Fig. 11.62 and Table 11.10 are also 
shown, as are the presumed, constant boundary conditions. 

Application of the proper operator at each of the interior grid points 
gives rise to a set of 18 equations in 18 unknown values of ¢. The 
coefficients of these equations are given in Table 11.11. It should be 
recognized that the transpose of this coefficient matrix is the unit-relaxa- 
tion table. Contour lines of the resulting solution are indicated in Fig. 
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11.66 by the solid lines, and by the values listed at each pivotal location. 
“More exact”’ solutions, obtained from an electrical analog of the problem, 


are shown dashed. 


11.7 Parabolic Partial Differential 
Equations: Transient Problems 


If convection and radiation are neglected, the temperature distribution 
along a uniform-cross-section wire of length L satisfies the parabolic 


partial differential equation 
6) og 
a zs (11.117) 


at? 


where u(x,t) = temperature of bar 
K = thermal diffusivity 
Initial and boundary conditions for such problems are frequently of the 


type 
u(0,t) ae fi(t) 


u(L,t) = fo(t) 
u(z,0) = g(x) 


(11.118) 


380 modern methods of engineering computation 


The corresponding two-dimensional space-time heat-conduction equation 
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In Sees. 11.5 and 11.6 it was demonstrated that for elliptic-type equa- 
tions the value of the solution at any one interior point depends on all 
of the boundary values. Such is not the case for parabolic-type prob- 
lems. Here, by assuming for illustration the one-dimensional space-time 
problems, the value of u(2,,t;) depends only on the values of f,(¢) and 
fo(t) in the range 0 <¢t<t,; This means that in the solution of these 
types of problems, a step-by-step or ‘‘marching”’ method can be employed. 
On the surface, then, it would appear that parabolic-type partial differ- 
ential equations should be easier to deal with than elliptic types. Unfor- 
tunately, this is not necessarily the case. Marching methods, by their 
very nature, introduce a difficulty not usually present in elliptic prob- 
lems, namely, numerical “instability.”” However, as will be demon- 
strated, these special problems can be overcome by a proper choice of 
the nodal increments. 


11.7.1 Explicit Formulation 


The essential features of the finite-difference procedures developed for 
elliptic equations can be extended to the solution of parabolic equations. 
For example, for the two-dimensional case (one space dimension and 
one time dimension), the grid system of Fig. 11.67 can be used. If it is 


assumed that the partial differential equation in question is 
EEE (11.120 
at ax? cae 


difference approximations for the first and second derivatives are required. 
If the second-order central differences in both of the coordinate direc- 
tions are desired, the necessary equations (from Table 9.27) are the 


Figure 11.67 
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following: 
, —s ok U4 U2) (112121) 
au 1 
be? hi (uz — 2uo + u1) (lis22) 


Substitution of these into the given differential equation yields 


1 
yet us) es — 2uo + uw) (5123) 


By solving for we, the explicit equation for continuing in the ¢ direction 
a numerical solution would be 


Us = 27. (us = 2uo +- U1) ++ Va (11.124) 
k At 
where ar (An)? (Pis125) 


The corresponding computational molecule is shown in Fig. 11.68. 
_ To examine the suitability of this molecule, consider the propagation 
of an error e, presumed to be introduced in the computational process 
at a single pivotal location. This error could be due to round-off or 
other reasons. Two cases, r = 44 and r = 1.0, will be examined. For 
both, to allow a comparison of the resulting values, Ax is presumed equal 
to 1.0. Clearly, from Figs. 11.69 and 11.70, the computational molecule 
defined by Fig. 11.68 is numerically unstable, for bothr = 1.0 andr = 1. 
Moreover, it is interesting to note that the magnitude of the error 
increases for a decrease in the grid spacing in the ¢ direction. 

Instead of presuming the central-finite-difference expression [Kq. 
(11.121)] for the first derivative of uw with respect to ¢, it will now be 
assumed that the most simple forward-difference expression is to be used. 
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Figure 11.71 


That is, 


as Se =f 11.126 
at = ae Uo U2) ( ) 


Again using Eq. (11.123) for the definition of the second derivative of 
u with respect to x gives the computational molecule shown in Fig. 
11.71. The error propagation for this molecule is illustrated in Figs. 
11.72 and 11.73 for r = 14 and 1.0, respectively. For r = 14 the error 
decreases with ¢. For r = 1.0 an increase is observed. 

Possibly a clearer indication of the error propagation associated with 
the molecule shown in Fig. 11.71 can be seen from a plot of the absolute 
values of the errors at the x location where the error was first introduced. 
(It is to be recognized that while the values shown on such a plot repre- 
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sent discontinuous points, an ‘‘envelope of errors’ can be drawn for each 
of the cases considered.) 

Figures 11.72 and 11.74 do not prove, but only illustrate, that the 
simple forward finite-difference expression with r = 14 yields a stable 
solution. It can be shown analytically, however, that solutions to para- 
bolic partial differential equations obtained by using the computational 
molecule of Fig. 11.71 will converge to the correct solution providing 
that ris fixed andr < 146. Moreover, it can be shown that such solutions 
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Figure 11.75 


are stable.{ A further advantage to selecting r = 14 can be seen in the 
computational molecule of Fig. 11.72. The multiplier for the 7, 7 
position is zero. Moreover, the value ¢(7, 7 + At) is the average of the 
two values ¢(7 — Ah, 7) and g(zi + Ah, 7). 

Computational molecules for use in the solution of three-dimensional 
parabolic problems (two spatial dimensions and one time dimension) 
can be obtained in a manner similar to that developed above. For the 
grid system shown in Fig. 11.75, the computational molecule is given 
in Fig. 11.76. For stability and convergence it is required that 


(11.127) 


+ A solution is said to converge if the difference between the approximation and the 
exact solution approaches zero as the size of the increment used approaches zero. A 
solution is stable if an error from any source diminishes as time increases. 


Figure 11.76 
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Figure 11.77 


Example 11.9 To illustrate the application of explicit-recurrence 
techniques, consider the numerical solution of the parabolic partial 
differential equation problem defined by Eqs. (11.128) and (11.129). 


2 
gee (11.128) 
ot One 
(Osi 06 w c2,a=) Oc. 0 eo (11.129) 


The solution domain is shown in Fig. 11.77. It is to be noted that since 
the initial condition ¢(z,0) does not vary with z, and since both “‘lateral 
boundary conditions” are equal, symmetry of solution about zx = 1.0 
is to be expected. The problem then can be reduced to a considera- 
tion of the case illustrated in Fig. 11.78. 

For the purpose of comparison, an analytical solution to the problem 
will be obtained first. If a product solution of the type defined by Eq. 
(11.130) is assumed, 


o = F(2)G() (11.130) 


then the given differential equation reduces to the following two ordinary 
differential equations: 


Ff” = —\°F and G’ = —NG (11431 
The general solution will be, therefore, 


¢ = (Ci cos \x + C2 sin \x)C3e-*t (11.132) 


Figure 11.78 


fee 
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To satisfy the boundary condition ¢(0,t) = 0 
CC 0 (11.133) 
To ensure that ¢’(1.0,t) = 0, that is, to satisfy the second boundary 


condition, 
cos Ax = 0 (11.134) 


or Net Gta) 3, 5.0.5 (11.135) 


For the initial condition ¢(x,0) = +1.0 (denoting C:C3; = C4), 


Ci sin S10 (11.136) 


n=1,3,5, . = 
1. nw 
or Cy=2f, sin Fede 


The analytical solution to the problem in question is, therefore, 


oo 


4 
o= y —- (sin is r) e— (nin? /4)t (11137) 
P#136, 0.0 “ 
The solution is shown graphically in Fig. 11.79. 
It will first be assumed that a four-equal-segment grid spacing in the 
x direction will result in a sufficiently accurate numerical solution. 


That is, Az will be selected equal to 14. Quite arbitrarily, the ratio 


Figure 11.79 
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r = At/Ax? will be taken as 1. The resulting value of Af is then 


LN 1 1 
At = (;) r= (=) (5) = 0.0104 (11.138) 


The grid system under examination is shown in Fig. 11.80. Also indi- 
cated is the corresponding explicit-recurrence computational molecule. 
It is to be noted that there is an apparent inconsistency in the listed 
initial (or boundary) value of ¢g at x = 0,t = 0. Along the z direction 
the required value is g = 1.0. The boundary condition for all values of 
tat x = 0 requires that ¢ = 0. The average value has been listed. 

An extra row of grid points outside the domain of interest has been 
introduced to facilitate the handling of the boundary condition 


g2(1.0,t) = 0 (11.139) 


From Fig. 10.138, it should be evident that such a boundary condition 
requires that 
¢9(0.75,t) = ¢(1.25,2) (11.140) 


Figure 11.81 shows the computational molecule placed correctly for 
the evaluation of ¢(0.25,0.0104). The computation is that illustrated in 
Eq. (11.141). 


[1] [¢(0.25,0.0104)] = (5) [0.5000] + () [1.0000] + (;) [1.0000] 
(11.141) 
or ¢(0.25,0.0104) = 0.9168 (11.142) 


Figure 11.80 


Boundary conditions 


04 Z, 90,0208 4 0.0312 0.0416 
O O 


LPL LLL LE LA LALLA LL 
10) 


(1.25)4----—- O- 4 ----- O------- ~§- —-----6--—-----~-------- 
(.og0e) Nis. va 2 
Imaginary grid introduced to allow 
realization of boundary condition 


~,(1.0,4)=0 
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Figure 11.81 


Values of ¢ for the range 0 < t < 0.1 are summarized in Hig. 11.82: 
It is to be noted that to facilitate presentation, the coordinate directions 
in this figure have been interchanged. That is, the variable t has been 
listed vertically, and x horizontally. To give an indication of the accu- 
racy of solution obtained, Table 11.12 compares the numerical values 
obtained from t = 0.1040 with corresponding values from the analytical 


solution [Eq. (11.137)]. 


0.1040 Oo ———— 0.4166 —— 0.7268 


Figure 11.82 
——x 
O 0.25 0.50 0.75 1.00 1.25 
] 1 
O 0.5000 1.0000 —— 1.0000 1] OOKE} —— te =a 10000 
ci, ee == 1.00/00) 1.0000 ——— 1.0000 —— 1.0000 
i 0.0208 i Sg — 0.9862 Peoee — 1.0000 —— eel 
0.0312 eee ae 2556 ————10 9959 O83 ——= me ee me ONE Mis 
0.0416 SS ee === O/IS2 OPNQIOS! = O22) 9/4! —— 0.9909 
0.0520 0 Bade ee OIG 5 ee60 ss ice ol 0.9800 
0.0624 a y = ne OES) = © Si = Oh ex) 
0.0728 0S alae == OHO) O94: 9 i(e—— sete === 0.9497 
0.0832 10) eee, = OATS OS8Z0 ——= OIA ——= 220 
0.0936 ees ce = 10) 74a) ecis == OBIS) EOS 
! | 0.8940 —— 0.9434 — 0.8940 
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Table 11.12 


¢ Analytical 0.4157 0.7261 0.8938 0.9428 
¢ Numerical 0.4166 0.7268 0.8940 0.9434 


Example 11.10 For the same problem, by selecting r equal to 14 and 
maintaining Az at 0.25, the increment in At is 


1 
At = (24)?(44) = a 0.03125 (11.143) 
Solution for this set of conditions is summarized in Fig. 11.83. Also 
shown is the corresponding computational molecule. A plot of these 


values is shown in Fig. 11.84. 


Example 11.11 Little additional difficulty is introduced when the 
differential equation also contains the first derivative of the function 


Figure 11.83 


O 0.25 0.50 0.75 1.00 1:25 

=e al chs T el 
7 O 0.5000 —— 1.0000 1.0000 1.0000 1:0000 ———= darts 
¥ 0.0313 07500 al 10000 — 10000 oc 1.0000 
0.0625 O 0.5000 0.8750 1.0000 1.0000 —— 1.0000 
0.0938 : ee eke Caen ace — 0.9375 
0.1250 i oares 4a See acne —_ 0.8750 
0.1563 i 0.3438 Aa eee eee Os — 0.8125 
0.1875 : eae ee eae Hees — 0.7500 
0.2188 ; epee = meee ee pee — 0.6954 
0.2500 ! opens ———= oes ne neuen — 06406 
0.2813 ; oan otdeas pises oeaoe 0.5939 
0.3125 l 0.2066 o1420! pleat GEES —. Aehes 


GY 
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Figure 11.84 


with respect to x. Consider, for example, the equation 


dp Oy deg 
— = — 4+ 4 — 11.144 
ot Ox? a Ox ( ) 


It will be presumed that the same boundary conditions hold as were 
prescribed for Example 11.9. That is, 


¢(0,t) = 0 g2(1,t) = 0 o(x,0) = 1.0 (11.145) 
By presuming the numbered grid system of Fig. 11.85, Eq. (11.144) can 
be written in different form, as given in Eq. (11.146). 


il 1 4 
At (—¢o + ¢2) = (Az)? (gs — 290 + 1) + 3(Ax) (—¢gs+¢1) (11.146) 


Figure 11.85 


At 
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Figure 11.86 
By defining 
ee (11.147) 
(Ax)? 
t 
and s= ae (11.148) 
Ax 


the computational molecule is that shown in Fig. 11.86. If 7 is selected 
equal to 44 and Ax = 14, then s will be equal to 14, and the computational 
molecule will be that of Fig. 11.87. Error propagation is examined in 
Fig. 11.88. Solution would proceed in the manner of the preceding 
cases. 


Example 11.12 To illustrate an “explicit”? numerical solution of a 
parabolic equation in two spatial and one time coordinates, it is presumed 
that a spatial region 0<2< 1.0, O<y< 1.0 is subjected to the 
initial condition defined in Fig. 11.89. The specific equation to be 
solved is 


— =— 4+ — (11.149) 


Figure 11.87 
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| | | 
O — 0.3955« O — 0.2637e O—0.0879« == 
| | | 
ee 00.426. O— 12109 O— 0.0469 
| 
9 — 0.4219 O— 0.4219€ 0 —0.1406« 00,0156. — 
| | | | 
————) O— 0.5625« O—0.3875« O0— 0.0625« Ou 
for | | | 
) ) O0—07500« O— 0.2500 ) ) 
| | | 
0 © 0 {00009 O 0 0 
| 
[ ) fo) ©) fo) ) O O O 
ee | | | 
Ax 
Figure 11.88 
*+=0 
A 


Direction of 
propagation 


~ 


x 


Figure 11.89 


Figure 11.90 


|= 
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#=0.0468 


7=0.0312 


Figure 11.91 


The boundary conditions for t ¥ 0 are 


z(0,y,t) = 0 
gy (2,0,2) = 0 
PCy.) = 0 (11.150) 
g(x,1.0,t) = 0 


By choosing Az = Ay and r = 14 (see Fig. 11.76), the computational 
molecule shown in Fig. 11.90 is obtained. Solutions for the first three 
increments in time are shown in Fig. 11.91. 
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7=0.0156 


11.7.2 Implicit Formulation 


The explicit method of solution described in Sec. 11.7.1 has two major 
drawbacks. First, to ensure stability, only very small increments in the 
time variable are permissible. This frequently leads to excessive com- 
putation. Secondly, as is evident from a consideration of any of the 
previously developed explicit computational molecules, ¢2,4a: is only 
dependent on ¢z-az,t; ¢z,t, ANd ¢z4az,. The boundary conditions at 
t= 1 are not involved. This is illustrated in Fig. 11.92. A more 
correct representation would require that the definition of g at any 
particular value of time, regardless of the x value of the pivotal location 
in question, also include the influence of the known boundary values 
at that same time. That is, all ¢ values for a given time ¢; should be 


396 modern methods of engineering computation 


Boundary 


we 
> 
~ 


Region within which 
boundary values have 
no influence — using 
explicit formulation 


rw 
iS as 


YW. 


uy Explicit 


computational 
molecule 


Boundary 


Figure 11.92 


interrelated—including the boundary values. Such an interrelationship 
is achieved by using an implictt-recurrence formula. 

An implicit-recurrence formula is one in which two or more unknown 
values of the function at ti41 are expressed in terms of known previous 
values, by a single application of the formula. Such formulations, then, 
require solution of sets of simultaneous equations. 

Implicit-recurrence formulas for parabolic systems can be most readily 
derived by presuming that variation in the x direction—in the finite- 
difference formulation of the given differential equation—can be repre- 
sented by weighted averages of formulations at both ¢; and ¢,+1 (see Fig. 
11.93). If it is assumed that the contribution at time ¢,,; is 6, where 
0 < 6 <1, the multiplying factor at é; must be (1 — @). For the para- 
bolic equation 

dp 070 


at az? (11:15) 


the implicit-recurrence equation is 
1 
A; ae a a (ie 6) ee aces 


1 
ab (—) (ys — 2¢2 + ys) (11.152) 
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or 


galr(1 — @)] + ofl — 2r(1 —9)] + gilr(1 — @)] 
= 95(—76) + ga(1 + 2r6) + os(—76) (11.153) 


where r = At/(Az)?, and0 < @<1. The computational molecule asso- 
ciated with Eq. (11.158) is shown in Fig. 11.94. Also illustrated are 
several special cases, including the previously defined explicit-recurrence 
formula. Implicit-recurrence formulas are stable for all values of r 
providing 6 > 14. The particular case where 6 = 1%, that is, the Crank- 
Nicolson formulation, is most frequently used. 

It should be recognized that formulation using the implicit-recurrence 
computational molecules of Fig. 11.94 results in solution of tridiagonal 
sets of simultaneous equations. These equations are usually well con- 
ditioned and readily solved by any of the methods described in Chap. 5. 


Example 11.13 To illustrate the general method of solution, consider 
again Example 11.9. The equation to be solved is 


ra) 2 
pees (11.154) 
ot ax? 
and the boundary conditions are specified to be 
et) = 0 —vgit0)=10 ~ 2) =0 (11.155) 
The following particular case will be examined first: 
6= 1.0 Ax = 0.25 At = 0.05 (11.156) 
Therefore 
St Fo 0.8 (GG eas) 
p= (any? , ; 


The computational molecule is shown in Fig. 11.95, where it has been 
presumed that the ¢ coordinate increases vertically downward. The 
notation that will be used in formulating the equations is that defined in 
Fig. 11.96. For the computational molecule placed in the fashion of 


Figure 11.93 


GS ® @ 


Ar 
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Figure 11.94 


Figure 11.95 
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O-- Os COMES. © 


0.25 0.50 OS 1.00 (1.25) 
ie °f0500 1000 [1.000 | 1.000 ‘1,000 | (1.000) 
| 
| 
| 
t 0.05 = — = 
@) e079 | eis Tee, 3,1 ~ 4,1 | (e5.1)=93\1 
| 
| 
(2) 0.10 = —— al 
~o,270 |?1,2 ~2,2 ~3,2 4,2 | (~5,2) = "3,2 
@) 0.15 LL | 
Po,3-0 
Figure 11.96 


Fig. 11.97, the resulting equation is 
—0.8¢01 + 2.6¢1,1 —0.8¢21 = 1.000 (11.158) 


For evaluation at ¢ = 0.05, the following equations must be solved: 


0,1 
SE Ne $11 1.000 
Ses, AY PS g2,1 | _ | 1.000 
== (\ a? Gee ().9 v3.11 | 1.000 (13-059) 
SIR PG S| en 1.000 
5,1 


By noting that go. = 0 and ¢3,1 = ¢3,1, these equations can be written in 
the more normal form 


2.6 —0.8 1,1 1.000 
—0.8 2.6 —0.8 2,1 1.000 
an 11.160 
—0.8 2.6 —0.8 || ¢3,1 1.000 ( ) 
—-16 2.6 4,1 1.000 
Figure 11.97 
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Solution is 


¢1,1 = 0.656 
go1 = 0.881 (11.161) 
Ose ta 0.956 
ga. = 0.973 


With these values, solution at ¢ = 2(0.05) = 0.10 can now be obtained. 
The controlling equations are: 


2.6 —0.8 1,2 0.656 
Sie oo Aa oS 0.881 
2) 11.162 
LiQise oo 6 Os Meee 0.956 |  ‘(it-162) 


It is to be noted that the coefficient matrix is the same as that of Eqs. 
(11.160). This will be true for all values of ¢. It is to be further rec- 
ognized that the solutions at any given stage become the constants in 
the simultaneous equations at the next stage. 


Example 11.14 For comparison, again with the assumption that 
r = 0.8, the Crank-Nicolson (@ = 14) formulation of the first set of 
simultaneous equations is given by 


1.8 —0.4 71,1 0.800 
—0.4 1.8 —0.4 $2,1 1.000 
—04 “18-014. fogs] ol 110007 |e 
—0.8 1.8 4,1 1.000 
Solution is 
Ol i= 0.648 
2,1 = 0.917 (11.164) 
¢3,1 = 0.980 
g41 = 0.991 


11.7.3 Method of Linest 


Also known as the Runge-Kutta method, the method of lines presumes 
representation of derivatives in the x direction by finite-difference apprex- 
imations, while maintaining continuous integration in the ¢ direction. 
To illustrate the general procedure, again consider the parabolic partial 
difference equation 


de 070 
Cie oe (11.165) 


An explicit formulation, assuming ‘ordinary’ finite-difference approxi- 


tit is to be noted that the method of lines is a technique commonly used to solve partial 
differential equations on an indirect electrical analog computer. 
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mations in the x direction, is 


0g:,; 1 
oa = (Ax)? (i143 — 20%,5 + its) (11.166) 


But for such assumptions, ¢ along each of the preselected pivotal lines 
is a function only of t. The partial differential equation can, therefore, 
be expressed as the ordinary differential equation 


dg; 
dt 


2 1 
+ ie (gs) — i (Gi-1 + ¢it1) = 0 (11.167) 


where i, = Mee 


In essence, then, the method of lines reduces the solution of the continuous 
system to a consideration of the behavior of sets of simultaneous linear 
first-order ordinary differential equations. 

Implicit formulation using the Crank-Nicolson procedure can also be 
extended to the solution of problems by the method of lines. For most 
of these cases, the Runge-Kutta equations of Table 10.5 are used for 
integration in the ¢ direction. 

It is to be remembered that the Runge-Kutta methods of Chap. 10 
can be used to solve nonlinear ordinary differential equations. The 
combined method of lines and Runge-Kutta method, implicit or explicit 
formulation, can therefore be used to solve nonlinear partial differential 
equation problems. 


11.8 Hyperbolic Partial Differential Equations 


The two methods most frequently used in the numerical solution of 
hyperbolic partial differential equations are (1) the method of char- 
acteristics and (2) the method of finite differences. The first of these 
is uniquely suited to the solution of hyperbolic-type systems. 


11.8.1 Method of Characteristics 


The general concept of the method of characteristics can be illustrated 
by considering the behavior of a general, single, quasi-linear, first-order 
partial differential equation. 


Go be =e (11.168) 


It should be noted that a, b, and c can be constants or functions of u, 2, 


and/or y, but not of du/dzx or du/dy. 
It is presumed at the outset that the equation in question corresponds 
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Figure 11.98 


to a single surface in a three-dimensional space whose coordinates are 
x, y, and u(z,y) (see Figs. 11.98 and 11.99). Moreover, from P to P’ 
on the surface in question, the change in the value of the function uw, 
that is, du, can be related to the rates of change in the function value 
with respect to the coordinate directions x and y. 

ce) 0 

du == teen (11.169) 

Ox oy 
Equations (11.168) and (11.169) constitute a set of two simultaneous 
equations in the variables du/dx and du/dy. 


Ou 
Oh Tale. CEA eh 11.170 
dx dy du ~ | du (11.170) 
oy 
Figure 11.99 
yh 
ds 
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If the determinant of the coefficient matrix associated with Kq. (11.170) 
is not equal to zero, the set will have a unique solution. However, if 


a ob 
2 a = 0 (11.171) 


there will exist an infinite number of possibilities. That is, an infinite 
number of surfaces could contain the prescribed line PP’. 

A curve along which the determinant of the coefficient matrix vanishes 
is called a characteristic. For the single first-order equation considered, 
the characteristic direction is defined by expanding Kq. (11.171). 


ady = b dz 
or ee (11.172 
De we) 
The slopes of such lines are 
d b 
se ae (11.173) 
dx a 


It is to be noted that these characteristic directions are determined inde- 
pendently of considerations of specific values of du/dx, du/dy, and c. 

However, no solution to the problem of the determination of u = u(z,y) 
for a given partial differential equation can be realized unless there are 
also required the compatible conditions 


“G@ € 

Pte 0 (11.174) 
and Cc bine 

ih vd = (W175) 


These equations result from substitution of the right-hand column of 
constants for either of the columns of the original coefficient matrix, 
requiring that the resulting determinant vanish. Expansion of Eqs. 
(11.174) and (11.175) yields 


Bs (11.176) 


The method of characteristics is concerned primarily with the deter- 
mination of the particular directions along which integration with respect 
to only one of the coordinate directions can proceed. 


Example 11.15 In the range 0 < x < 1.0, 0 < y < 1.0, solution is 
desired to the partial differential equation 


du, du 


— = 1.0 (A777) 
Ox Oy 
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The prescribed initial conditions are 


u(z,0) = (1.0 — xz)? u(0,y) = 1.0 — y? (11.178) 
In terms of the notation of Eq. (11.168) 
a=b=c=10 (11.179) 


From Eq. (11.173), the characteristic direction is 
d b 
BSL aii (11.180) 
i ae 
and the equation corresponding to characteristic lines is 
dy = dx or y = x + (const), (11.181) 


With Eqs. (11.176), the function value obtained by integrating along the 
characteristic is 


du = dx or u = x + (const). (11.182) 

For the specific case illustrated in Fig. 11.100, 
Uu— U = © — Zo (11.183) 
where Zo = 0.20 and Uo = 0.64 (11.184) 


It is to be noted that variation in u along the characteristic is linear. 
Moreover, only the initial value u,(0.20,0) = 0.64 influences the value 
of the function along this line. 

The complete range of solutions is given in Fig. 11.101. 

Consider next the following two partial differential equations: 


Tiss Weg 3 f = f(uyr,x,y) 
(11.185) 

ES ts ree ee) 

Ox ay at UV, L,Y 


Figure 11.100 


a 
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u(x,0) 
0.64 — 


Figure 11.101 


It is to be noted that two dependent variables u and v have been pre- 
sumed. Two surfaces are, therefore, in question (see Fig. 11.102). Two 
expressions for the differential changes in the function values are, there- 
fore, required. Equations (11.185) and (11.186) constitute two sets of 


te) 
da + dy = du 
0 oy 
(11.186) 
ov ov 
—+d—=d 
ie Ox 7. oy 
Figure 11.102 
u(x,y) 
y 
on 
du 
5 


ds 
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two simultaneous equations in the variables du/dx and du/dy, and du/dx 
and duv/dy. 


| 4] oY a (11.187) 
dx dy J| du du 


and ie a a -\7 (11.188) 


Each of these sets of equations will define a characteristic direction. 
Each will result in an equation for integration along that characteristic. 
For Eqs. (11.187) 


a 6b - 
dz fe “= \dx <dy| Cle 
dx ay au 
eee 113i 
or ‘ 5 j (11.190) 
For Eqs. (11.188) 
Colm) Cam; 
ie a aie Map ee 0 (11.191) 
an dy ids 
or 7 Gas (11.192) 


Example 11.16 The equations to be examined are 


la i de ity 11.193 

Ox oy ‘ dx ay eee 
The initial conditions, which must be specified for both uw and v, are 
presumed given only for the case y = 0. 


u(x,0) = 0.50 v(z,0) = 1.00 — x 


From Kags. (11.190) it is determined that the direction associated with 
one of the families of characteristics, say the a characteristic, has a slope 
with the x axis of. +1. 


dx = dy = du (11.194) 


or an sini (11.195) 
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Integration along that direction requires that 


u . eons (11.196) 
Using Eqs. (11.192), 
hip a 
he (11.197) 
and —dz = dy = dv (11.198) 


yk 


Figure 11.103 


The two characteristics a and 6 can be combined to determine the values 
of wu and v at all their various points of intersection. This is shown for 
one specific case in Fig. 11.104. Also listed in that figure are initial 
values of wu and v at Ax = 0.2 intervals. 


Along a characteristic: 

%— 19 = yf — or u — 0.50 = 0.10 or u = 0.60 
Sees (11.199) 

Along 8 characteristic: 


V—V=Y—Yo or v—040 =0.10 or v=0.50 


Figure 11.104 


O Ox2 0.4 0.6 0.8 1.0 
u(x,0) = 0.50 0.50 O50) OX) Cae) Tousi0) 
v(x,0) = 1.00 0.80 0.60 0.40 0.20 O 
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The more general set of two first-order partial differential equations is 


) ) 0 ) 
rp ae) = aes 2 Sod oe ‘i 
Ox oy Ox OY 
(11.200) 
Ou Ou Ov Ov 
A= BS C— p= F 
Ox 7 Oy #8 Ox i oy 


where a, b, ¢, d, f, 4, B,; C, D, and F can be functions)of v4, 2, 
or y, or constants. The full 4 X 4 set of simultaneous equations to be 
examined is given as Eqs. (11.201). 


Ou 
Ox 
G20 aCe Ou f 
Al 1B Cel) ay F 
dx dy 0 O |] g| | du (11.201) 
0 0 dx dy]}} 5, dv 
Ov 
oy 


The characteristic directions are obtained by equating the determinant 
of the coefficient matrix to zero. 


OP EDS tC aed 
Alba le 
hes ee Ne = 0 (11.202) 
Oe Oe ccs 


Expansion yields 


dy\? dy 
(s4) (aC — Ac) — — (bC — cB + aD — Ad) + (0D — dB) = 0 


(11.203) 
For the determinant 
Gut) Cameo 
Al eI = Ca) ie 0 
Pe i hI S (11.204) 
0 d dx dy 


d 
du | op Fiat = (eC & Ac) | + dv(cD — Cd) 


+ dz | Fa De “ GC = Fe) | =0 (11.205) 
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For the case where 


CEOs, Cea if, 
As B C F 
0) = Gke ch 


du(—aB + Ab) + dv i — Be + a (—aC + Ac) | 
+ | By — bF + “ (Fa — Ap| =0 (1.207) 


It is to be recognized that for hyperbolic systems Eq. (11.203) defines 
two characteristic directions. Integration of either of Eqs. (11.205) or 
(11.206) in these two directions will result in the desired solution. 


Example 11.17 


ou 
Ox 
tO EO pean 1.0 0 
dx dy 0 -@-|| 91 | du (11.208) 
0 O dr dy || a, dv 
dv 
dy 
From Kq. (11.203), substituting the corresponding values fora, b, ... ,f 
and A, B, ... , F from Kq. (11.208), 


dy\? dy 
aoe a (0 a 7 () 
(2) am -2£@+4+-% 
dy re 
or = = + Vn? = tn (11.209) 
Similar substitution into Eq. (11.205) yields 


du (- #) + dv(n) = 0 (11.210) 
dx 


The total solution is, therefore, that summarized in Table 11.13. 

Along a boundary (see Fig. 11.105), the set of initial conditions given 
in Table 11.14 is prescribed for the set of equations listed as (11.208). 
n = 2.0 is also specified. To determine the values of u and v at location 
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Table 11.13 


a Characteristic B Characteristic 


dy\ _ 25, \ eee 
Cee | (e)-= 
du(n) + dv(n) = 0 du(—n) + dv(n) = 0 


or or 
utov =ct u—v = Cot 


yc: and C2 are constants, depending on the 
initial conditions. 


P16 (see Fig. 11.105), 


“+0 = €; = 1.60 from point @) 
; gene. (11.211) 
u—v=C, = —1.40 (from point ©)) 
or u = (ce. + @) = 0.10 v = W(a — ce) = 1.50 (11.212) 


Solutions at all possible intersection points are listed in Table 11.15. 
The points of intersection are indicated in Fig. 11.106. 

Consider, now, the solution of a second-order quasi-linear hyperbolic 
partial differential equation. In its most general form 


d*9 0°y 079 
a Se 
Ox? Ox dy Oy? 


eal (11.213) 


where a, b, c, and f can be functions of x, y, 9, d¢/dzx, and d¢/dy, or 
constants. To facilitate representation, the following standard notation 


Figure 11.105 
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Table 11.14 
ee ee er ee ee ee 
Location: @ @ ® @ © © 
ee ee ee 
uU 1.00 1.00 0.90 0.80 0.70 0.60 
v 0.10 0.20 0.40 0.80 1.20 2.00 
(1 =ut+yv 1.10 ie20) 1.30 1.60 1.90 2.60 
C2 =uUu-—v 0.90 0.80 0.50 0 —0.50 —1.40 
Table 11.15 
a Characteristics 
B Char- 
acteristics oS co a3 a4 as a6 
C—O 6 S120 Cr — oO Mier — IeGO)licy 91.90) c1 — 2.60 
Bi 
co = 0.90 
Bo AOS 
C2 = 0.80 Je OES 
Bs ue 0-280 TOES) 
C2 = 0.50 M— O30 v— 0935 
Bs te — OND u = 0.60 u = 0.65 
co = 0 a») =-0155 v = 0.60 » = 0.65 
Bs i VAD u = 0.35 u = 0.40 a = O95 
C2 = —0.50| v = 0.80 v = 0.85 v = 0.90 — 05 
Bs wu = —0.15}u = —0.10| uw = —0.05}| wu = 0.10] wu = 0.25 
¢2 = — 1740) w= 1.25 ja—wieo) ewe) = 1250)» = 1.65 


Dennen EEE Enna 


will be presumed: 


_ 4 
on dx 
meee 
Gage ay 
070 
= — 11.214 
Re ( ) 
079 
gy = 
Ox OY 
0” 
—— 
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> xX 


Figure 11.106 


In terms of these quantities, the required differentials are 


du =pdx+qdy (112215) 

dp =rdx+sdy (11.216) 

dq = sdx + t dy (11.217) 
and the original differential equation is 

ar+bs+ ct =f (11.218) 


Equations (11.216) to (11.218) can be written in matrix form in the 
variables r, s, and ¢. 


a “0c r i 
dx dy O||s| =| dp (11.219) 
OF dard at dq 


By proceeding as before, the characteristic direction is determined by 
equating the determinant of the coefficient matrix to zero. 


a ON = 
dx dy 0|=0 (11.220) 
0 dx dy 
dy\? dy 
or a 3) b TE +c=0 (11221) 


The equation is hyperbolic if 6? — 4ac > 0. That is, Eq. (11.221) will 
have associated with it two “real” solutions for dy/dz. 
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Substituting the right-hand column of constants for the last column 
of the coefficient matrix and again equating the resulting determinant 
to zero yields 


CaO 

dry dy ap — 0 (11.222) 

Op dt dg 

dy 

or Cae a dq +adp — fdx =0 (11223) 
Substitution in the second column yields 

Coe 

dx ip 0 = 0 (11.224) 

0 dq dy 

dy 

or c dq + a dp —fdy=0 (ils 225)) 


Example 11.18 For the region 0 < x < 1, ¢ > 0, the simple wave 
equation 


dg .— Oy 
See = es 22 
ot? On? ( ») 
is specified. The initial and boundary conditions are 
g(x,0) = 4e(1 — x)x 
¢g(0,t) = 0 
we C1227) 
¢(1.0,t) = 0 
q(z,0) = ¢(z,0) = 0 
By recasting the given equation in the form of Kqs. (11.213), 
2 2 
be we deg) (11.228) 
Ox? ot? 
the coefficients a, b, c, and f have the following values: 
oi 0 vee 1 f = 0 (11.229) 
From Eq. (11.221), the characteristic directions are 
dt\? dt 
—} (1) —-— (0) -1 =0 
() w -20 
Po (11.230) 


or dx 
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p=4e(1—x)x 


Figure 11.107 


Along each of these, by using Eq. (11.225), integration would proceed 


according to 


or 


The solution is summarized in Table 11.16. 


Table 11.16 


a Characteristic 


dz}a 


ye [1 (4)| + O(dt) = 0 


dt 


=—d 
dx P 


dq 


8B Characteristic 


dt 
dx ] g 


dq(1) — dp(1) = 0 
or 
FSP eC 


dg(1) — dp(—1) = 0 
or 
qt+p=Cz 


(11.231) 


= (0.2 interval for 


numerical computation is presumed, the initial values at the preselected 


pivotal location are those listed in Table 11.17. 


It is to be noted that 


since ¢ is specified at ¢ = 0, p along this boundary is determined by 


differentiation. 
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Table 11.17 
Location 7 =O 4 SUP Pe =O! x = 0.6 cans eo = 10 
¢g 0 0.64e 0.96e 0 .96e 0.64. 0 
qd 0 0 0 0 0 0 
p 4.00e 2.40« 0.80e —0.80e —2.40e —4 .00e 
G =F) Gi —4.00e —2.40e —0.80e +0. 806¢ +2 .40e +4 .00e 
qtp=Ce +4 .00e +2 .40e +0.80e —0.80e —2.40e —4 .00e 


0 fe) 
ee ear) Ac. 22) (11.232) 
Ox Ox 


Similarly, along the boundaries x = 0 and x = 1.0, 


(Keo to) 
=. =— (0) =0 11.233 
= == ©) (11.233) 
At location P, gq = 0 and p = 2.40c. At Q,q =0 and p = 0.80c. At 
location R, 


eC) a 2 Ae (11.234) 
q+ p = (cx)p = +0.806¢ 
This gives 
= (ce. + 2) = —0.806e 
qd Wy(a 2) (11.235) 
p = 16(c1 — 2) = —1.606e 


Figure 11.108 


i an 
Cre =2)40« = 0/80 Along a characteristics 


Co= +2.40€ + 0.80€ Along B characteristics 
— +t 
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At location S, it is known from the @ characteristic originating at 
location Q that 
q+p = +0.80e (11.236) 


But the boundary condition at S, ¢(0,t) = 0, requires that q along « = 0 
equal zero. Therefore, 


+ps = +0.80¢ (11.237) 


This in turn requires that the new a characteristic (from S to VU) have 
the value 


q—p =90 —0.80e = 1 (11.238) 


In a similar manner, the values of g and p= associated with location 
T are 


q = 0 
q—p = —2.406¢e (11.239) 
or p = 2.406¢€ 


The 6 characteristic from T to U is controlled by the equation 


q+ p = 2.40e = c (11.240) 
At location U, 


6 (Cia + Co) = 14(—0.80e + 2.40c) = 0.806¢ 
(11.241) 


One = Cog) = 16 —0.80e — 2.406¢) = —1.60e 


It is to be recognized that ¢ values at the various points of intersec- 
tion are obtained by analytical or numericai integration of q (that is, 
d¢/dt) or p (that is, d¢/dx) values, starting from the boundaries. 


Example 11.19 In all the preceding examples of this section, linear 
equations were presumed. Moreover, constants that result in easily 
integrated functions were selected. The method of characteristics works 
equally well for nonlinear problems. However, for most of these latter 
cases, integration is carried out by means of approximate numerical 
methods. To illustrate a procedure that might be followed, consider a 


second-order nonlinear partial differential equation of the type defined 
in Eq. (11.242). 


dp I~ dy 
+c 
Oy? Ox Oy Oy? 


=f (11.242) 


In general, the equations for the characteristic directions will vary with 
x and/or y. Curves (rather than straight lines) will normally result. 
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Three equations are of importance: 
dy\? dy 
—) —b—+ é 
a (2) A c (11.243) 


di 
cdg +a— dp — f dy 
dx 


l| 
S 


I 
co) 


(11.244) 


du = pdx + q dy (11.245) 


If it is presumed that Eq. (11.243) has two real roots a and B associated 
with each set of values of x and y, the values of xr, yr, Gr, Dr, and yr 
associated with location R (see Fig. 11.109) can be determined from the 
following set of five nonlinear algebraic equations: 


Ye — Ye = |Yolor + apie — xe) (11.246) 
yr — Yo = [36(Br + Be)] (tr — XQ) (11.247) 


(qr — gp)[}o(cr + cp)] + (pe — pr)|o(aror + apap)] 
— (yr — yr)[6(fre+fe)] =0 (11.248) 


(qr — Qe)[¥o(cr + cr)] + (pe — pe)[o(arBe + aeGo)] 
— (yr — yo)l}4(fe + fo)] =0 (11.249) 


Ur — Up = [}6(pe + pp)|(te — Zp) 
+ [16(ar + qe) (ye — yr) = 9 (11.250) 


The solution procedure is normally an iterative one. 


Two factors concerning initial values and the use of the method of 
characteristics in the solution of hyperbolic systems are to be noted: 


Figure 11.109 


\ 
\ 
N 


\e8 characteristic 


2 


> xX 
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First, discontinuous initial values will propagate into the solution domain 
along characteristic lines. No undue complication is introduced by such 
a condition. Secondly, when the boundary along which initial condi- 
tions are specified coincides with a characteristic curve for the equation(s) 
in question, the values of the initial conditions must be identical to the 
integrated solution along the characteristics. 


11.8.2 Finite-difference Method 


While the method of characteristics is the most accurate and general 
of the several methods for solving hyperbolic partial differential equa- 
tions, the finite-difference method can be used to solve many of the 
problems that arise in practice. Again, as demonstrated in Sec. 11.7 
on solution of parabolic equations, two finite-difference formulations are 
possible: explicit and implicit. Again, pivotal spacing must be selected 
to ensure stability. 


Expiicir FORMULATION 
For the particular hyperbolic equation (the one-dimensional wave 
equation) 


= (11.251) 
using ordinary second-order central differences (see Table 9.27), the 


computational molecule is that shown in Fig. 11.110. For convergence 


eh T1252 
s=— , 
Nap oe 


For optimum accuracy, the selected grid coordinate ratios, that is, 
At/Azx, should be such that the region of finite-difference solution coincides 


Figure 11.110 


late 7 
‘i At 

oc 
A? 

At y 
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Age \ 46s NN nee 
; eo Gees SEES 


KY 7  ANAAN 


Wil 


Figure 11.111 


with the region of solution defined by the a and 6 characteristics of the 
given equation (see Fig. 11.111). 
Example 11.20 The given equation is 
Vp dy 
Se 1 43) 
ot? Ox? ( ) 
The initial and boundary conditions are 


g(x,0) = 4(1 — x)(@) 


¢(0,t) =0 
(11.254) 
g(1.0,t) = 0 
gr(x,t) = 0 
The region in question is 
Ox 1 and t>0 (11.255) 


If a square grid is presumed, that is, s = 1.0, the computational molecule 
for continuing a solution is that shown in Tig. 11.112. To handle the 


Figure 11.112 
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Figure 11.113 


special problem associated with obtaining the first set of solutions at 
t'= At, it is to be recognized that the last of the four conditions specified 


in Eqs. (11.254) requires that 


g(x,At) = o(x, — Al) 


The computational molecule for that special “‘first’’ case is shown in 
The solution through one full period is given in Fig. 11.114, 


Higaiield3: 
presuming that Ax = 0.2. 


Figure 11.114 
if 


0 ——+0.48 +0.80 +0.80 +0.48 O 

O——— 9 O18 ee a os / 

O——+ 0.48 mee ae pe ! 

1) + 0.116 <a eae na ! 

QO = OS fe = i O 

mare ee me seer ! 

Sure Ape ae es 5 st salle 

joie ve ke Bn dei 

0.6 ae eed oe oe ! 

0.4 ae ay : er mes cae | 

Or2 ee Sah ae a er: | 

0) : ie eee Pe a : 
O 0.2 0.4 0.6 0.8 1.0 
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Impuicir FORMULATION 

If it is assumed that the finite-difference expression for 02¢/dz? at loca- 
tion ¢ = t; can be represented by the average of the second-order finite 
differences at t = ¢; + At and ¢ = t; — At, the Crank-Nicolson computa- 
tional molecule for the equation 


htt 070 
— = 0 11.257 
at ax? ae 
is that shown in Fig. 11.115. Solution using this formulation will con- 
verge, independent of the value of s. 


Prob. 11.1: Develop higher-order computational molecules for V? and 
V‘ in the following coordinate systems: 

(a) 30° skewed (b) 45° skewed 

(¢)) Trianeular (oi 30°98 = 60°) (d) Polar 

Prob. 11.2: For the square region L X L, it is specified that values of 
gy on the boundary are those illustrated in Fig. 11.116. If V?’e =0 
within the region, determine the values of ¢ at the pivotal locations, 
presuming grid spacings equal to 

(a) L (b) BL ce) ML Od) YL 

Prob. 11.3: If, for Fig. 11.116, derivatives of ¢ normal to each edge 
are also required to be equal to zero, and if V‘y = K is specified for the 
interior of the region, determine the values of ¢ at the pivotal locations, 
presuming grid spacings equal to 

(a) eb (6) Le) ML) KL 


Figure 11.115 


Ly 


4 


Ax x 
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Figure 11.116 


Prob. 11.4: Determine the values of Y at the five pivotal locations 
indicated in Fig. 11.117. The interior of the region is controlled by the 
equation 

VY=vVW 


WV on all external boundaries is equal to zero. 


Prob. 11.5: Determine the first three eigenvalues for the circular 
plate with hole, whose edges are simply supported, as shown in Fig. 
11.118. The controlling equation is 


V2w + Aw = 0 


where w is the lateral deflection of the plate. 

Prob. 11.6: A triangular plate (plan view shown in Fig. 11.119) is 
fixed along its longer edge, that is, w = 0 and w, = 0, and simply sup- 
ported along each of its equal-leg shorter edges, that is, w = 0 and 


Figure 11.117 
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Figure 11.118 


Wy, = 0. The equation governing lateral deflection of the plate is 


q 
Wilh) es 
D 


where q is lateral load per unit area, and D is the plate flexural rigidity 
constant. Determine the maximum deflection of the plate, solving the 
problem by presuming three different grid spacings and extrapolating 
these results to an infinite number of pivotal locations. 

Prob. 11.7: Develop group-relaxation operators similar to those shown 
in Figs. 11.48 to 11.50 and 11.52 for 30, 45, and 60° skewed-coordinate 
systems. 

Prob. 11.8: Develop computational molecules similar to those listed 
in Table 11.10 for unequal grids, presuming patterns having spatial 
dimensions related by factors of 3. 

Prob. 11.9: For the region and grid spacing defined by Fig. 11.120, 
given the equation 


Véw + ee 0 
e One 
the boundary conditions are 

w(0,y) = 0 
w(1.0,y) = 0 

w(z,0) = 0 
w,(z,0) = 0 
W2(0,y) = 0 


Wee(1.0,y) = 0 
Wyy(x,1.0) = 0 
Weer(t,1.0) + 2wWryy(x,1.0) = 0 


Determine the first three eigenvalues associated with the system. 
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Fixed edge 


Ih, 
K = 
Figure 11.119 


Prob. 11.10: Discuss the numerical stability of the explicit finite- 
difference formulation of the equation 


Oz 
Ve z= 2(2z,y,t 
A (x,y,t) 


presuming ordinary second-order finite differences, Ar = Ay, and 


Prob. 11.11: The vibration of a beam is governed by the equation 
d4w Ze rw 0 
dct © ott 


For the initial and boundary conditions 


w(0,t) = 0 
w( 1.050). =) 

w,(0,t) = 0 
w=(1.0,é) = 1.0 


define the behavior of the beam, through at least one period of vibration. 


Figure 11.120 
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Prob. 11.12: By both the method of characteristics and the method of 
finite differences, solve the following four partial differential equations 
in the range 0 <x < 10,0 <y < 1.0: 


(eX) Ou = 

Or OY aa 

) 

Ox oy 7 
F) 
meer y= 44,() 
oy 
F) 
eS Bau 
oy 


The initial and boundary conditions are 


g(0,y) = 1.0 
V(0O,y) = 0 
u(x,0) = 0 
v(z,0) = 0 


Prob. 11.13: Show that Eqs. (11.190) and (11.192) are special cases 
of Eqs. (11.203), (11.205), and (11.207). 
Prob. 11.14: 


The initial conditions are 
v(z,0) = (1 — 2)? 
u(0 < x < 0.5, 0) = 1.0 
Oo az 10,0) =i 
Determine u(z,y) and v(z,y) in the range 0 < y < 1.0. 
Prob. 11.15: 
Gy 
a l ot bs fea 
1 LZ du Uu 
Ox 


For the special case where a = 1 and L = 4, determine the values of 
y and u in the range 0 <a < 1.0, and 0 <t< 2.0. The initial condi- 
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tions are 
¢(0,0) = 1.0 
g(x # 0,0) = 0 
u(a,t) = 0 
Prob. 11.16: 
Ou 
a +v=0 
av, au _ 
Ox Oy 
u(0,y) = 


Oy a TS ee 
Determine u and v in the rangeO <x < 10,0 <y < 1.0. 


optimization 


In its most elementary and basic form, optimization is con- 
cerned with the determination of the “‘best’’ solution to a 
given problem. This process is required in the solution of 
many problems in engineering and applied science—in the 
maximization (or minimization) of a given function(s), in 
the selection of control variables to facilitate realization of a 
desired condition, in the scheduling of a series of operations 
or events to control completion dates of a given project, in 
the development of optzmal layouts of organizational units 
within a given design space, etc. It is not the intent of this 
chapter to develop all or even a majority of the optimizing 
methods that have been proposed, but rather to describe 
several of the more often referred to systems. 


12.1 Introduction: Concepts and Definitions 


It is to be recalled that in those cases a function f(x) is said 
to have a maximum value at x = aif 

Aft =f ah) = fia) <0 
and Af- = f(a —h) — f(a) <0 


(12.1) 


427 
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for small values of h. For the case where Aft and Af~ are both positive, a 
minimum is realized. 

By assuming that f(x) is continuous in the interval in question, and 
moreover presuming that it possesses continuous derivatives through 
order n, Eqs. (12.1) can be expanded in Taylor series as indicated in 
Eqs. (12.2). 


1 
Aft = filayh + = 7"(ah? ef 
ss (12.2) 


1 1 
Af = -f@h+s f'n — sah + --- $0 


UT 


It is to be noted that a maximum (or minimum) can be realized without 
having f’(z) = 0. Consider the series of function(s) shown in Fig. 12.1. 
In the range 0 < x < x, a continuous function @) is assumed to exist. 
In the range x. < x < a, the function has been noted as @). At x4 
and x. a discontinuity in the function is presumed. (3) and @ are con- 
tinuous functions in the ranges x4 <x <x» and x» < x < 2s, respec- 
tively. Consider each of these regions independently : 


1. 0 <x <a: A (local) maximum occurs at 2, where f’(z:) = 0 and 


the smallest feve"(71) < 0. A (local) minimum value of the func- 


Figure 12.1 
| 
nN 
| 
| 
l 
S he 
~ | | 
XN | | | 
Peery ener 
| 
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| 
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tion in the interval in question occurs at the boundary value 2», 
where f'(x) ¥ 0. 


DORE r St eta A where f’(a2) #0. A 


(local) maximum is realized at the boundary value 2. Here, 
again, f"(w.) * 0. At a there is a point of inflection; that is, 
t Avs) = 0. 

3. 24 <x < xe: A (local) maximum is observed at 25, where f’(xs) = 0 
and the smallest feve"(z;) <0. A (local) minimum exists at the 
boundary 2, with f’(z.) # 0. 

4. %§ < xX < 4g: e = 27 is the location of the (local) minimum value of 
the function. Here, f’(v7) = 0 and the earliest feve"(x;) ¥ 0. 


It is to be noted that in each of these ranges, the local (or relative) 
maximum and minimum values have been noted. The absolute maxi- 
mum and minimum values for the entire range 0 < x < 2s are obtained 
by comparing the numerical values of these local extremes. 


-S(@)max = f(ts) — f(©)min = f(z) (12.3) 


For many problems in engineering and applied science it is possible to 
establish at the outset that for a particular interval in question, the 
function possesses one hump (or one depression). This property is 
known as n mathematical terms, unimodality requires 
the following: 


Mort, ts <0: J ATR ai 
(12.4) 
More? <7 2, <7 To ie iy 


In Fig. 12.1 the described function is unimodal in the interval x; < x < 2. 
The concept of a convex function is more restrictive than that of 
unimodality. If three values of the independent variable t, < x < x, 
are considered, convexity requires that the true value of y» be greater 
than that which corresponds to a linear interpolation between yq and ye. 
That is, 
(yea Gh aoe a: 


Yb Be Ya =F Yc (12.5) 
Le = Le Ue — Xa 


The type of reasoning used in the above-described single-variable 
case can be extended to the solution of problems having a larger number 
of independent parameters. For that particular case where n = 2, that 
is, f(a1v2), the behavior can be represented by a contour plot of equal 
values of the dependent function f. Consider, for example, that par- 
ticular case described in Fig. 12.2. The function is said to have a maxi- 


+ x* is the value of x corresponding to the maximum value of y = y*. 
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Ze Aa bere 


Figure 12.2 


‘S) 


mum at v7 — @ and 2 — bat 


fa@th,b +k) — f(a,b) <0 (12.6a) 
and a minimum if 

fiath,b+k) —f(a,b) >0 (12.6b) 
for sm of h and k. 


(12.7) 


where subscripts denote derivatives, for example, f,, = af/dz,. For the 
point a, b to be a stationary point, the first derivatives with respect 
to the independent variables must be zero, that is, f,, = 0 and f;, = 0. 
As was true in the single-variable case, to determine whether this sta- 
tionary point is a maximum, a minimum, or a saddle point, it is necessary 
to examine the second derivatives. A local maximum occurs when 


Sey, << 0 
a (12.8) 
and J ei) oes r= Ghee 0 
A local minimum oceurs when 
j fryz, Be 0 
(12.9) 
and Youatl eae acs Bes: a 0 


Tor saddle points, 
Fey, F ogy =A Gans = 0 (12.10) 
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With reference to Fig. 12.2, location @) represents a local maximum, as 


(12.11) 


By using the method of Leitmann,f{ a local maximum will be realized if 


D, > 0 Neel One. c, (12.120) 
or De <a) Me eB, Ope (12.126) 
where 
Le eee 7. ee Fiesty 
Da Te (12.13) 
Fe, Sines Joe 
For a minimum 
1 Se Ps IES an (12.14) 


Again, it must be recalled that maximum and minimum values can also 
occur at locations of discontinuities in the given function(s) or on 
boundaries. 


Problems in the general theory of maxima and minima are classified not 
only according to the number of independent variables involved but also 
as to whether these variables are interconnected by some other rela- 
tionship or subsidiary condition. For example, the problem may be to 
maximize f(x1,%2), subject to the constraint (21,42) = 0. Such a case 
is illustrated diagrammatically in Fig. 12.3. It should be evident that 
one direct method for solution of this problem would be to solve the equa- 
tion v(21,%2) = 0 and obtain a relationship between x; and x2. Sub- 
stitution of this relationship into f(z1,72) will reduce the maximization 
problem to one of a single independent variable. 

In general terms, the constrained maximum (or minimum) problem 
can be expressed as the maximization of the function 


f = F(t1,%2,%3, . . . Fa) (12.15) 


+ G. Leitmann, The Optimization of Rocket Trajectories: A Survey, in S. F. Singer (ed.), 
“Progress in the Astronautical Sciences,” app. A, North Holland Publishing Company, 
Amsterdam, 1962. 
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val Constrained maximum A (iV 
bd 
Jl 
eo” 
AC TRES, ws 
Maximum (no constraint ee 
ee 
— 
x4 
Figure 12.3 
subject to 
1(X1,02,03, 1 = » tx)=_0 
Gal, Los) « ata) OU 
Ora (12.16) 


oe #0 6 ies) eh © Wee wb leleiemye fel (eine 


CrlCiptets, sa ea 


If the m simultaneous equations (12.16) can be solved for m of the 
independent variables, these can be substituted into Eq. (12.15), thereby 
reducing the maximization (or minimization) problem to one of n — m 
independent variables. 


f= flr, 22,83, . -). ;La—m) (L2EK) 


Unfortunately, it is often extremely difficult (if not impossible) to obtain 
explicit expressions for m of the variables contained in Eqs. (12.16). 
For these types of situations, solution is most often realized by using the 
method of lagrangian multipliers. This method will be described in 
detail in Sec. 12.2. 

A second general type of constrained maximum (or minimum) problem 
is found in control theory. Here the problem is to maximize the function 


ft ="] Gets, Ga ec) (12.18) 
subject to the m constraining equations 

COM Ca Re Gene ge cies Ve) 

Chi Caws, «0. pa U 


(12.19) 


ame, BO ee Rwy tes el Es) fey te (eae See. 6 


Om UijXo,Uag ss age) = O 
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and the n equations in p unknown a’s 


W1(21,X2,03, «3 «© yhnyQ1,2,@3, ... ras) = 
W2(11,22,%3, o % « 9hm,Q1,Q9,03, . . a ae) = 0 
Boog 1h 0 ONGOING ot Gt OH Rene a neers ee (12.20) 
W,,(21,%2,23, + + « yhnyQ1,Q2,Q3, . aie 52s) = 0 
The problem is to determine the values of a1, a2, a3, . . . , ap Which maxi- 


mize the value of f. 

A third type of constrained maximum (or minimum) problem corre- 
sponds to that case where the functional relationships indicated in the 
second of the cases just described vary with time, and the function to be 
maximized corresponds to the final values of the independent coordinate 
variables. 


12.1.2 Other Classifications 


In addition to classifying optimization problems (1) according to the 
number of independent variables involved and (2) as to whether or not 
the optimization is ‘constrained,’ it is also necessary to categorize 
problems as to whether or not they are (8) statzc (i.e., time-independent) 
or dynamic (i.e., time-dependent). The last of the described three 
constrained maximization problems is time-dependent, whereas the first 
two are presumed to be static. Still a fourth type of classification is 
desirable: (4) when no unknown factors are present, the system is referred 
to as deterministic. For a system containing experimental errors and/or 
other random factors, the problem is stochastic. 


Before closing this section, it is to be noted that whereas in all the 
above it has been more or less presumed that the functions to be optimized 
are algebraic and/or transcendental, many problems require that integral 
functions be optimized. These quantities depend not upon the values 
of » discrete variables, but rather upon n complete functions. The 
problem then is to determine a function (or functions) which maximizes 
(or minimizes) the integral(s). Again, constraining equations may be 
specified. These general types of problems are discussed in Sec. 12.3. 


The lagrangian-multiplier method is directly applicable and ideally 
suited to the solution of constrained maximum and minimum problems. 
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1 L1,00;tan ee 


2(X1,%2,X3, Goo 9 


dowom Ows (12.22) 


Da Liseayay ee 


ae. f ene eae eet 4 (12.23) 
It is to be noted that the function f contains the origina unction to be 


maximized, f, each of the constraining equations ¢;, and m undetermined 
(so-called lagrangian) multipliers ),. 


That is, 


of 01 Oe 003 Om 
seb EE he = 
ae aN F a> Ne any + rz F — a al 0 
af Be a a om 
$$ eS ps St tO 
OX OX, 0X2 OX» OX (1235) 
of O¢1 O¢g2 03 OGm 
Xr ON h\e=—= 3 nie a 
ae ac i rs igen aN aE 0 


¢1 = 0 


(12.26) 


Example 12.1 It is desired to determine the relative cross-sectional 
areas of the two members in the two-bar pin-connected truss shown in 
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Figure 12.4 


Fig. 12.4 which result in the le 
straining condition is that 


The con- 


of the truss. 


t is to be pre- 
sumed tha e members are of uniform cross section throughout their 
lengths, though they may—and probably will—differ one from the other, 
and that stresses are within the elastic limit of the material. 

The cross-sectional area of member AB is A;. For member BC, the 
cross-sectional area has been denoted as A». Since the weights of each 
of the members are proportional to their cross-sectional areas times their 
respective lengths, the objective function to be optimized (minimized) is 


W (Ai,A2) = Ci(rf/2 Ay 4 A,) (12.27) 


where W = total weight of truss 
Ca = pL 
p = density of material 
By using any of the methods of deformable solids, the deflection is found 
to be 


(12.28) 


7 a? 4/2 cos? 6 , feos @ + ain Q)? 
Ed) a A; 


is would be written as 


In the normal form fo 


6h 
where C, = — (12.30) 
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By proceeding as in Eq. (12.23), the “new (nonconstrained) objective 
function” to be minimized is 


or Ww = C(V2 Ay + A») 
apes (cos ain | a 
AC, AsC 


The necessary conditions for optimization are, therefore, . 


(12.33) 


and (12.35) 


Solution of these three simultaneous equations for the necessary design 
parameters yields 


A.C, = V2 (cos 6)(3 cos 6 + sin 6) (12.36) 
A.C. = (cos 6+ sin 8)(3 cos 6 + sin 6) (12.37) 
These are plotted in Fig. 12.5. 


Example 12.2 Find the minimum distance from the origin to the 
curve 


y =a? — 2x4 +2 (12.38) 


. Figure 12.5 


5.0 


A,C2 
and 
LG: 


Least weight design 
Deflection constraint 
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Here, the problem is to minimize the function 
fzyla=e+y? (12.39) 
subject to the constraining equation 
e(a,y) = 2? — 24 +2-—y=0 (12.40) 


If the constraint is incorporated into a new function to be optimized, the 
augmented equation comparable to (12.23) is . 


f=ftrAae=22 + y? + (a? — 22 —y +2) =0 ~~ (12.41) 


The necessary conditions for a minimum are, therefore, 


of = 24 + A\(Qxz —2) = 0 (12.42) 
Ox 
ye 2y + (—-1) =0 (12.43) 
oy 
af 

and ann aul ee ce ee (12.44) 


From Eas. (12.42) and (12.48) 


» 


d 
oy Eres and ia 9 (12.45) 


_ Substituting these values into Eq. (12.44) yields 
Mw — 3 —4=0 (12.46) 


The one real root of this cubic equation is e 
dh = 2.196 (12.47) 


which when substituted into Eq. (12.45) yields 
= 0.687 and y = 1.098 (12.48) 


This, in turn, gives the desired minimum distance as 
min f(z,y) = V2? + y? = V1.678 = 1.295 (12.49) 


To illustrate that the solution given as Eq. (12.49) is, in fact, the 
desired minimum, consider the two curves of Vig. 12.6. The first is a 
plot of the constraining equation. The second, which uses the paired 
values of x and y from the first, gives the relationship between the coordi- 
nate dimension x and the distance r from the origin to the constraining 
curve. 
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Constraining 
equation 
V=rX2-2x4+2 


240) 


2 = —> 
os x 1.0 AO) Ka O 1.0 BAO) 2S 


Figure 12.6 


12.3 Introduction to the Calculus‘of Variation 


The calculus of variation is primarily concerned with the determination 
of extreme values of functions defined by integrals whose integrands con- 
tain one or more functions assuming the role of arguments. That is, the 
problem deals with the determination of the extremum of a function F 
of functions. Or, in still other words, the calculus of variation is con- 
cerned with the establishment of ‘an infinite number of points, all of which 
identify a curve, a surface, a hypersurface, etc., depending on the nature 
nd all of which 


To illustrate the concept, consider the integral 
(12.50) 


where z is the independent variable, y is an unknown function of x, and 
F(x,Y,Yx)Yrr) 18 a known function of the variables x, y, yz = dy/dz, and 
Yor = a’y/dx?. (The term functional is frequently used to describe 
functions defined by integrals whose arguments themselves are functions. ) 

It is to be presumed at the outset that the function y(x) is continuous 
throughout the interval in question, a < x < b, and is differentiable at 
least through one more than the order noted in F. It is further assumed 
that the unknown functions y(x) take on at the boundaries x = a and 
x = 6 the known values y(a) and y(b), respectively. 

If it is prescribed that y = y(z) is the correct solution which results in 
an extremum (a maximum or a minimum) for [(y), then it is possible to 
consider a second function 


G(x) = y(a) + en(x) (12.51) 
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which results in a greater value for the integral. is a small, real param- 
eter, whereas n(x) is an arbitrary, continuous function defined in the 
interval a < x < bso thatthe function itself and its first derivative with 
respect to x vanish on the boundary at x = a and x = b. Two possible 
curves are illustrated in Fig. 12.7. It is to be recognized that in the limit 
the second functional, I(y + e), must reduce to the correct solution. 


That is, 


6 
Ty + en) = [Flay +n ys t+ ete Yer + eee) dz (12.53) 


dl b 
yy = DS aaa a ae LEME 
€ €= 6 a 
oF oF oF 
where fF, =— FP, = Yen = (12°50) 
Y OYx OYax 


It is to be noted that as « approaches zero, the first variation of the 
dependent function y(z) is 


oy ==Y— y = ef - (12.56) 
Similarly, 
% Olas cen; 
é (oe) 
and bY re = Yaz 


Multiplying Eq. (12.54) by e and substituting these values yields 


oi i (Fy dy + Fy, yz + Fy,, 522) dx = 0 (12.58) 


yh 
ae 


440 modern methods of engineering computation 


Integrating, by parts, the last two terms gives 


b b d = 
[P Gy, dye) de = (Fy, yk — fo F,) by de (12.59) 


b Wie : b a? 
J Poe Bee) de = Ul uel, =| a.) dy | + 225 Pan) by 
(12.60) 


But n(x,y) was selected so that n(a) = n(b) = 0 and n2(a) = n2(b) = 0. 
Therefore Eq. (12.58) can be written as 


b d ile 
éf = ip (7,- 27, +h.) dy dx = 0 (12.61) 


However, 6y can be any small, arbitrary value different from zero. 
Therefore, the integral can be zero only if the expression contained 
within the parentheses equals zero. 


Equation (12.62) is the necessary eulerian differential equation associated 
with the variational problem 


I(y) 
| where : I(y) 


min 


(12.63) 


CATE Ripe eke 


(12.62). 
Example 12.3 The potential energy U of a beam of length L, uniform 


cross section—having a bending stiffness per unit of length of HJ—and 
subject to the loading shown in Fig. 12.8, is given by 


EI ie 3 
U= i, E (Yes)? — 3 CE = a (1 — *) | dx (12.64) 


Figure 12.8 
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It is required to determine the differential equation corresponding to the 
equilibrium state. 


For the illustrated case the boundary conditions are y(0) = y.(0) = 0 
and y(L) = yzz(L) = 0. 

Since it is known that equilibrium corresponds to that particular y(z) 
which minimizes the total potential energy of the systems, 


sU =0 (12.65) 


will result in the desired solution. 
With the notation of Eq. (12.50), the function F is 


EI P ae 
| F= ey (Yzx)? = 2 (yz)? = Ww (1 i *) y (12.66) 
Therefore 

oF x 

fF, = a a) (1 ae *) 
aF 3 

ts => —_— = —P(yz) (12.67) 
OYx 
oF 

a a — EI (Yzz) 
OY xx 


Still further differentiations yield 


C5) =a = Ay) 
/ (12.68) 


and (Fy...) ce EI (Yzzzz) 


d 
dx 
d? 
dx? 
Substituting these values into the Euler equation gives the following 
fourth-order ordinary differential equation: 


EI (Yzz2z) =f P(Yzz) =i) (1 - i) 


Cie ae Pod Fw ( *) 

— + — — = —(1—=— 12.69 
oF dzt * BI dx? EI aa 
This equation can be solved by any of the regular methods of ordinary 
differential equations, or by the numerical procedures described in Chap. 
10. It is to be recognized that the solution must satisfy the given 
boundary conditions. 


The procedures developed above for the solution of functions of one 
independent variable can be extended to other cases. Consider, for 
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example, the solution to the problem of minimizing the integral 
I(u) = JF @,y,u,us,u,) dx dy (12.70) 


In this case u = u(x,y) and uz = du/dx and u, = du/dy. 
Again, assuming a correct solution w(x,y) which minimizes the inte- 
gral in question, 


I(u ot en) = SIF, Y, U + €N, Uz =F 69; Ug a €ny) dx dy (12.71) 


OF 
or = ne — ~ bu +2 ae + — 6u,]} dr dy = 0 (12.72) 
J We OUy 


Possible surfaces are illustrated in Fig. 12.9. If it is required that 
n(z,y) = 0 on the boundaries, Eq. (12.72) reduces to 


oF a aF > 0 OF 
Hee E- a ) ou dxdy=0 (12.73) 


du ax OUz OY OUy 


(12.74) 


Table 12.1 summarizes for various integrals the eulerian differential 
equations which yield stationary values of functionals. 


Example 12.4 Show that a function that minimizes the integral 
Es Par cay 2 7e) 
is the solution of the biharmonic equation 


Wi = Users Lllcegy ot ugg (12.76) 


Figure 12.9 
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Table 12.1 Eulerian Differential Equations Which Yield Stationary 
Values for Various Integrals 
a a ee | eee 
Eulerian Differential Equations 


Int 1 Equations : 
a Corresponding to 6J = 0 


One Independent Variable 


OF oF OF OF 
Ty) = SF(@,yr,yays, . - - Yn) dx - = = =—= 
OY OY2 OY3 OYn 
oF d f{ oF 
I = fh ayy.) a = () 
(y) = SF(z,y,yz) dx anita: (=) 
oF d [{ oF d? { oF 
if = fF YG, Yr Yar d Ane ae ee =1() 
(y) = JF (@,y,Y2,Yex) dz a Ghee (=) “seat (*) 
NOES 5 3) WHC ee nae oF ad / oF d? { oF 
See a Praag Yael =0 
Ya@2) - - » Yen2er,---)AE\ gy dz \ ay, dz? \ dyz2 ; 
oF d [ oF dad? [ oF 
a =f ='0 
dz dx \ 0zz Ot INO zee 
Two Independent Variables 
oF 0 oF 0 oF 
= — = @ 
Tu) = fJP (eyjujtayty) de dy wos (=) a (=) 


GD)  TWE Geis Nis oF 0 [{ OF 8 oF * O2n aor 
UrzUryUyy) dz dy| au aa \ duz dy \ dUy 0%? \ Ure 


o* oF o? OF 
+2 = = 0) 
Ox Oy \OUzy Oy? \ OUyy 


Vo = "Ue Uy (12.77) 


Since 


_ the function F in Eq. (12.75) is determined by expansion. 
Te dae dye" ice? Ap 2g y te Uyy) A, OY 
or = Ue + Ly or yy (12.78) 


From Table 12.1, it is noted that the Euler equation required to mini- 
mize I (that is, 6J = 0) is 


OF 0 (OF 7) (+)+2 (=) 
Ou Ox OUz dy \Ouy OL? \OUrs 


2 2 
ty ae (“)+5 (=) =o (12.79) 
) 0707: \OUz, dy? \OtUyy 
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But for the functions listed as Eq. (12.78) 


OF 
a Pe eee eee i =F (12.80) 
Ou OUx OUy Oley 
oF 
ju = 2Uzz a QUyy = 2(Uzz [ Uyy) 
“ (12.81) 
oF 
= Des ta Athen — 2 (tes = Uyy) 
OUyy 


Therefore the Euler differential equation yields 


3? e2 0 
aio See (OG wl = 
ax WAGE + Ua) Se AUP [ (u Simms v)] 


or VA =U eee Ue (12.82) 


Constrained extremum problems also exist in many areas of engineer- 
ing and applied science. The constraints may be either algebraic, differ- 
ential, or integral equations in form, or they may be a combination of 
the three. 

lor the special case where n independent variables yi(x), yo(x), y3(x), 

. . , Yn(x) prescribe a function 


b 
L(Y1,Y2,Y3, - + + Yn) = if F(x,Y1,Y2,Ya, - + + Yn) dx (12.83) 
which is to be optimized, and which is subject to the constraining equations 
b 
J, = ip Di (L,Y 1,Y2,Y3, SoS ho Yn) = 0 
b 
J» = ap Bo(x,Y1,Y2,Y 3, eg Ya) = 0 
(12.84) 
b 
Jan a i Bn (L,Y1,Y2,Y 3 OS Yn) = 0 


the problem can be solved as a succession of ordinary minimum prob- 
lems. The procedure will be the same as that described in Sec. 12.3. 
A new augmented integral J first will be defined which contains m unde- 
fined, constant lagrangian multipliers. 


To= [eb ids Oa oc helt ae ee ee (12.85) 


From this 


t= : b 
min I = i. (F + Ny; + A2P» + A3P3 -/- cag ge + dmP m) dx (12.86) 


But since, by definition, the y;(x)’s are not interrelated, optimization of 
the integral can be handled as a series of ordinary minimum problems 
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for each value of x froma tob. The necessary conditions are 


fe) 
a (F + By + AvBe + AZhs + - + > + AmBm) = 0 
1 
fe) 
ip (F + Ari + Asbo + Ashs + - > + + AmB) = 0 
He COD 
fe) 
oye oe a Pa ee er isi\ en e— 0 


For the more general case where the y,’s are not, by definition, inde- 
pendent one from the other but are interrelated through integral con- 
straining equation(s), the following more general procedure must be 
followed. The functional to be minimized is, for example, 


b 
SOR = i F(a,yiyi) dx tae A peal ig Oe ea) (12.88) 
The constraining equations are 
b 
J; = iP ®;(x,Yyi,yi) Ax LS 2S ee yh Ue PS Ay dy By) Sian: eg MN 
(12.89) 


[In both of these equations the dot sign denotes a derivative with respect 
to the independent variable; that is, y¥; = (d/dx)(y:).] 

To facilitate solution there first will be defined the augmented function 
F’, which is composed of the original F plus the sum of the products of a 
set of m lagrangian multipliers \; and the constraining equation (12.89). 


le Mel N Biter ot olor Baek (12.90) 
or Fo= F + yi®i + doPe + As®s + + + + + AmPm (12.91) 


The necessary condition(s) for solution follow directly from application 
of the appropriate Euler equation 


fe) a \. @ 
(F + d,®;) — =| (fe ve) a0) (12.92) 
OYi dx | OY: 
Were =e oes, wo and 7 = 1, 2,3, 04. <9, M. 


When the integral to be optimized is 
[Peg Fea Wes alr (12.93) 
and the constraining equations are 
oj; = 9o;(2,yi) = 0 (mld, Onan tn) eal aye Og... , 1 (12.94) 
the augmented function F is defined as in Eq. (12.95). 
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Pha pope ape ly 2d) ee (12.95) 


Here, u; is a lagrangian-type multiplier, which, instead of being a con- 
stant, is a function of the independent variable. That is, 


Bj = Bi(X) (12.96) 
The necessary conditions for solution are therefore the appropriate Euler 
equations, 
- £ (5) <0 ee ek onl Ae) (12.97) 
OY; VAL Noy; 


and the constraining equations (12.94). 
For the still more general set of equations 
T(y,2) = f- P@y2vth2) ax (12.98) 


with the constraints 
p(X,Y,2,Y,2) = 0 (12.99) 


the Euler-type equations required to ensure 6/ = 0 are 


) d for dg d dg 
pre D  acscoergedd Vere bees Poms Se Se Weg 
oy dx \oy oy dx OY 


a d (dF dee sdefandc 
and 0z ) dx () E dz dx (u f) 


These in combination with Eq. (12.99) provide a sufficient number of 
conditions to define y, z, and uw as functions of x. 


os) 


(12.100) 


Example 12.5 A flexible rope of uniform cross section and length L 
hangs in equilibrium under the influence of gravity. Its two ends are 
attached to two fixed points A and B, as indicated in Fig. 12.10. The 
problem is to determine the equation of the curve assumed by the rope. 

If g is the acceleration of gravity and p is the mass per unit length 
along the rope, the potential energy of the entire rope with respect to 
the datum y = 0 is 


Iy) = fy (eady as (12.101) 


The constraint required is that the integral of the ds elements equal the 
total length of the rope. That is, 


L 
L = f, as (12.102) 
By noting that 


ds = V (dx)? + (dy)? = dx i + (4) (12.103) 
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Figure 12.10 
Eqs. (12.101) and (12.102) can be written as 
b 
Ty) = eg fy. + y22) de (12.104) 
b 
and Pe i: (1 + y,2)* de (12.105) 
To ensure minimization, the Euler differential equation to be solved is 
1 + y,2)4 — - 2) 
KE ye) cE la ty 0 (12.106) 
Integration yields the desired equation of a catenary: 
We 


y = + (f) cosh 


(12.107) 


To facilitate evaluation of the constants i, ¢, and &, a new location of 
the y coordinate axis, one which bisects the distance ba, will first be 
presumed. (This is shown as y* in Fig. 12.10.) Since it is specified 
that points A and B have the same y = h values, 


(a+ a see ey, +(a roe sce 


or €=0. Solution of Eq. (12.105), using the value for y listed in 
Eq. (12.107), yields 


hk-— \ = (£) cosh (12.108) 


L = 2(f) sinh (12.109) 


(a + b)/2 
g 
From this, with Z and (a + 6)/2 known, ¢ can be determined. 


Example 12.6 Considering the family of curves that lie wholly on a 
given surface S and pass through points A and B, determine the differ- 
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ential equations required to minimize the are length from A to B. (See 
Fig. 12.11.) 

The total distance on the surface between the two given points A 
and B is 


je ie VJ xe + ye + oe at (12.110) 


where each of the coordinate dimensions is related to a fourth (inde- 
pendent) parameter t. That is, c = x(t), y = y(t), andz = 2(t). Since 
the arc must lie on the surface S, the constraining equation is 


S(z,y,z) = 0 (12.114) 


The presumed boundary conditions correspond to ¢ = a and t = b. 
Proceeding as described in Eq. (12.95) gives the augmented function 
to be optimized as 


F=F+4.z8 (1271.12) 
where F = (a2 + y? + 22)” 
w= w(t) (12.113) 
S = S@,y,2) 


Equations (12.97) yield the following three equations: 


os ad Zt 
u— a) =O (12.114) 


Eliminating u from each of these gives the desired two equations 


d (*) d (i) d € 
a\e) a\r) avr 


a8 SS 
Ox Oy 0z 


(12.115) 


Figure 12.11 
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These, together with the particular equation of the surface (L211), 
allow solution of the problem in question. 
If the surface is a sphere whose center is at the origin, the constraining 
equation is 
S(a,y,z) = 2? + y2+22-—a =0 (12.116) 


and Eqs. (12.115) reduce to 
Fiu — uF: FY tt a yl Fey — “F; 


a = oF? Feige G2A17) 
By rearranging terms, these can be written as 
YXLte — LY te = F, = 2Ytt — Yet (12.118) 


Ye — Lz Ji ZY: — Ye: 


Considering only the first and last group of terms in this equation and 
integrating twice yields 
x—py+qz=0 (12.119) 


which is the equation of a plane through the center of the sphere. That is, 
the minimum arc lies on a great circle through points A and B. 


12.3.1 Ritz’s Method 


While boundary-value and other types of problems can be described in 
terms of integral expressions which have minimum values for those 
specific functions which solve the particular problems in question, many 
do not admit of any rigorous solution. For those which do, it has been 
demonstrated that solution is equivalent to obtaining answers to appro- 
priate Euler differential equations. However, an equally important use 
of the principles that have been developed relates to the construction 
of sequences of approximate functions which converge to the desired 
Euler solutions and therefore to the resolution of an appropriate varia- 
tional problem. Ritz’s method is one such method that presumes that 
_ the solution can be approximated by a linear combination of suitably 
chosen functions. 

To facilitate description of Ritz’s method, it will be presupposed that 
the exact solution for a particular extremum problem corresponds to 
y(x) = y*(x). That is, [(y*) corresponds to, for example, a minimum 
value of J. If a new “admissible function” g(x) is substituted into the 
functional equation, 


LG) 21 G*) (12.120) 
Moreover, if a convergent sequence of functions 


G(x) = 91 + G2 + ee a oe OR 
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is selected such that 7, (x) contains the complete family 


Gut G2 t+ * 3 ei 
it is reasonable to conclude that 
lim Gn = y* (12.121) 


Ritz proposed the construction of a linear combination of functions of 
the general type 
y(x) a ~(2, 01,42, ovis’ sax) (12.122) 


where ¢ is such that for all values of a; the end conditions are satisfied. 
G1, Qo, . . . , G& are real parameters. 
Substitution of Eq. (12.122) into the functional equation to be opti- 
mized yields 7 
[(a1,42,43, . . . ,@%) = 0 (12. 12a) 
which can be handled directly, as described in the earlier parts of this 
chapter. That is, 
cia 


Vid ip Sb ies ye oat: 212 
an j (12.124) 


results in minimization (or maximization) of the now algebraic function 
I. It is to be noted that the sequence of families of functions is 


yi(x) = ¢1(x,a1) 

y2(x) = g2(x,a1,d2) 

ys(X) = ¢a(X,G1,02,d3) (12.125) 
Yn(2) = On(X,01,42,43, aa Qn) 


Convergence to the correct solution depends primarily on the selection of 
“relatively complete sets of functions” for the y’s. Two relatively com- 
plete sets in the interval 0 < x < f, which satisfy boundary conditions 
y(0) = y(£) = 0, are 


n 


G(x) =o) axt(t — 2) (12.126) 
k=1 
2 . kra 
and 9(z) = »: Qi ONES, (12.127) 
k=1 


It is to be noted that the Ritz method requires that only the ‘geometri- 


cal boundary conditions” (y and yz) be satisfied by the approximating 
function. 
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Example 12.7 By using Ritz’s method, the extremum of the 
functional 


Ty) = f) (ys) = ay) dex (12.128) 


subject to the boundary conditions y(0) = 0 and y(1.0) = 0, is to be 
determined. 


Assume as an approximating polynomial the following power series: 


n 


G.(x) = » (1 — x)(a,2x*) (12.129) 


k=1 


It is to be observed that both boundary conditions are simultaneously 
satisfied by this choice of functions. Table 12.2 summarizes several of 
the terms required in the solution. 


Table 12.2 

n Yn Yn, 

1 (1 — x) (a2) ai(1 — 22) 

2 (1 — x)(aix + aox?) ai(1 — 2x) + ae(2x — 32?) 


3 (1 — x) (aia + aex? + asx?) ai(1 — 2x) + a2(2x — 3x?) + a3(8x? — 423) 


Consider the first possible case, n = 1: Substitution of the appropri- 
ate terms into Eq. (12.128) yields 


1 
Ig) = ff {lax — 20)! — alae — 2*))} de (12.130) 
Integrating and substituting in the limits give 
Ij) = Yar? — Yom (12.131) 


Since this equation is an algebraic expression in the variable ai, it is 
possible to handle the minimization in an elementary fashion 


al 
61 =0 = —— = %ai — Hn = 0 ale) 

Oa, 
<, 1 aa (12.133) 


The corresponding function for 9:(x) is then 


yi(x) = (1 — 2)(%x) (12.134) 
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For the case where n = 2, the functional is 
1(g) = fp {lax(1 — 20) + an(2x — 32*)]* 
— zlai(x — x?) + ao(x? — x*)]} dr (12.135) 
Integrating and substituting in the limits yield 
1(G) = Var? + VYara2 + 24542” — am — Mode (12.136) 


For 6J = 0, 


al (y 
1D) 36a, + Yas — M2 = 0 
‘ (12.137) 
oly 
and D) sg, + $4502 — }40 = 0 
0a2 
Solving these simultaneously gives 
a1 = Yio a2 = Me (12.138) 


The equation for the approximating function is therefore 
G2(x) = (1 — x) fox + }402") = Hol — x)@— 2’) (12.189) 


Additional terms can be taken into account; however, it 1s to be recog- 
nized that the number of required computations increases at an increas- 
ing rate. 


12.3.2 Galerkin’s Method 


In general, Galerkin’s method employs a so-called ‘‘semi-inverse’’ pro- 
cedure for solution of extremum problems. A differential equation is 
first presumed (or known) which corresponds to a Euler equation for 
an appropriate functional. For example, if the differential equation 
selected is 


d 
Ly) = p(x)ly] — [g(x)}-— °F [r(x)yz] = 0 (12.140) 


where p(x), q(x), and r(x) are presumed to be functions of x or constants, 
it can be demonstrated that the functional being optimized is 


I(y) = JF(x,y,yz) dx (12.141) 
where Pe= Foray. a op ay” — qay (12.142) 
In Example 12.4 it was shown that solution of 


Vij =U ee) (12.143) 
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is equivalent to maximizing (or minimizing) the functional 


Ty) = SJ (Vu)? dx dy (12.144) 
For the more general nonhomogeneous fourth-order differential equation 
Viu = f(x,y) (12.145) 

the functional is 
I(y) = Js((V?u)? — fu] dx dy (12.146) 


It is to be recalled that the basic concept of the Galerkin method 
requires that the error associated with an assumed polynomial be orthogo- 


nal to a set of linearly independent functions ¢;. (See Sec. 10.4.) 
That is, 


2 ends, <0 (12.147) 


where e = error due to substitution of trial polynomial into given differ- 
ential equation 

suitable coordinate functions (which satisfy all the boundary 
conditions associated with the differential equation in question) 
The trial polynomial is presumed to be of the general form 


Gi 


y= Ze AiLi (12.148) 


where the a,’s are constants. 

Substitution of the finite polynomial described in Eq. (12.148) into 
the given differential equation L(y) = 0 will ordinarily result in a value 
other than zero; that is, there will be an error. 


Ly) = e€(x) « #0 (12.149) 
The general Galerkin method [Eq. (12.147)] therefore requires 


lowe agi) Peete, ioe ean (12150) 
j=l 


The subscript change within the parentheses from 7 to 7 is to emphasize 
that the error is a function of the total polynomial assumed, whereas 
orthogonalization takes place independently with respect to each of the 
coordinate functions. 


Example 12.8 Using Galerkin’s method, again determine the 
extremum of the functional 


Ly) he [(y2)” — xy] dx (12.151) 


subject to the boundary conditions y(0) = y(1.0) = 0. The associated 


Euler equation is 
L(y) = yer + x = 0 (12.152) 
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The selected polynomial approximation (which satisfies the given bound- 
ary conditions, term by term) is 


je = Y (L — 2)(ui2*) (12.153) 
k=1 
By considering just the first term in this series, 
Yi = ax(1 — =) (12.154) 
the error is 
LG) = Gx + Wx = —2a, + r= e (12.155) 


The coordinate function is determined as follows: 


fi = aig: = a{x(. — z)] 


or gi = «(1 — x) (12.156) 
The necessary Galerkin equation is 
iB (360-2 Wee 2) de (12.157) 
which yields 
For the case where two terms are presumed, 
G2 = ayx(1 — x) + aex?(1 — 2) (12.159) 
the error is 
€ = a1(—2) + ao(2 — 62) + bx (12.160) 
and the coordinate functions are 
gi = «(1 — x) (12.161) 
and go. = x7(1 — 2) (12.162) 


Here, two Galerkin equations are required: 


1 
ij [ax(—2) + a2(2 — 62) + Wal[x(1 — x)] dx =0 (12.163) 
1 
i [a1(—2) + a2(2 — 6x) + La][x2(1 — 2)) de =0 (12.164) 
From the first of these, 
8a, 4a, = 1 (12.165) 
From the second, 
20a, + 16a, = 3 (12.166) 
Simultaneous solution yields 
a= oe a2 = Yo (12.167) 


These are exactly the same values that were obtained by Ritz’s method. 
This is not a coincidence. For many (if not most) variational problems 
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in engineering, Galerkin’s method (with the assumed coordinate func- 
tions satisfying all the prescribed boundary conditions) and Ritz’s 
method (with only “essential” boundary conditions being met by the 
coordinate functions) result in the same solution.t Moreover, it is to 
be recognized that Galerkin’s rules for obtaining the coefficients of the 
coordinate functions are simpler than Ritz’s rules. 


12.4 Direct vs. Indirect Methods 


By definition, an indirect method for optimization is one that is based on 
a “necessary condition’ for that optimization. A direct method, on the 
other hand, depends on a “direct”? comparison of function values at two 
(or more) points. To illustrate the basic difference in these two 
approaches, consider the problem of determining the maximum value 
of the single-independent-variable problem 


y = f(z) = maximum (12.168) 


An indirect approach would be, for example, to determine the value of 
x for which yz = 0. The direct approach is concerned with establishing, 
by comparison of various y values, the particular value of x for which y 
isa maximum. All the methods previously described in this chapter are 
indirect methods. 

When analytical solutions are desired, indirect methods are preferred. 
For individual cases, however, direct methods are more often used. 
Direct methods have a further advantage: end points and points of 
discontinuity in the function value present no additional computational 
difficulties. It is to be recognized also that as the number of independent 
variables increases, the direct method has a further computational 
advantage: the indirect method requires the evaluation of at least n 
equations, whereas the direct method requires the successive evaluation 
of only one equation. 

Direct methods for optimization are most often referred to as seek 
methods or search methods. For single-variable problems, where it is 
known that the function is unimodal, these procedures have been exten- 
sively developed, and it is possible to prescribe patterns of search that will 
result in the definition of known “intervals of uncertainty” within a 
minimum number of trials. A few of the more often referred to single- 
variable search techniques are the Fzbonacci search, the golden section, 
lattice search, etc. Multidimensional search problems are not so clearly 
understood. Nevertheless, usable procedures such as steepest ascent 

+L. Collatz, ‘‘The Numerical Treatment of Differential Equations,” p. 208, Springer- 


Verlag OHG, Berlin, 1960. (Translated from a supplemental version of the second 
German edition by P. G. Williams.) 
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(or descent), sectioning, pattern search, and others have been developed 
to handle these types of situations. Section 12.5 is concerned speci- 
fically with single-variable search problems. Section 12.6 deals with 
the direct method of solution of multidimensional problems. 


12.5 Single-variable Search 


Throughout this section it is presumed that the particular function being 
examined is unimodal in the interval in question. [See Eqs. (12.4).] 
Moreover, it is assumed that sequential search procedures—those pro- 
cedures where information from one trial can be used to benefit the 
location of subsequent trials—are to be developed, rather than simul- 
taneous ones—where all trials are run at once. 

To define the general problem and illustrate the procedure that will be 
used in its solution, consider a range of independent variable 0 < x < a. 
Nothing is presumed known in advance concerning the function in 
question except that it is unimodal. Assume now that the function 
value at a particular value x = 2x, is determined (for example, by evaluat- 
ing a given function or by carrying out an experiment). This is shown 
in Fig. 12.12. With only this much information known, it is impossible 
to determine whether the maximum (or minimum) value of y occurs to 
the left, to the right, or at x1. Therefore, the “interval of uncertainty” 
in the location of that particular «* for which y is a maximum is the 
entire interval from x = 0 tox = a. 

If now a second trial is carried out at, say, x = x2 > “1, and if the 
resulting value y2 = y(%2) > yi = y(21), the interval of uncertainty can 
be reduced from that previously established. The definition of uni- 
modality [Eqs. (12.4)] rules out the possibility that ymax can exist to the 
left of z;. Therefore, ymax must exist within the interval 2; < x* < a, 
as shown in Fig. 12.13. Should a third trial xz. < x3; < a@ result in a 
value of y3 less than y2, it can be similarly reasoned that ymax must be 
contained within the interval x; < x* < 23. This is shown in Fig. 12.14. 


Figure 12.12 
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It is the purpose of this section to develop procedures for progressively 
and methodically reducing the interval of uncertainty, and thereby allow 
the determination of x* and ymax with a minimum number of trials. 


12.5.1 The Search Plan 


Tor the problem just described, depending on the values of the y’s, it 
was possible to select new trial values of x which allow a progressive 
“‘geroing-in’’ on the desired optimum value. It is to be recognized, 
however, that no real, analytically defined, repeating pattern of search 
was followed. 

A more meaningful search plan would be one that allows an a priori 
determination of the effectiveness of the search: a plan that is not depend- 
ent on the values of the particular dependent function being optimized, 
one that is concerned only with defining the range of x within which 
the optimum will occur. 

In developing such a search plan, it must be recognized that at any 
stage in the process the optimum value of the dependent variable will 
be contained within a “one-division x interval” on either side of the then 
known maximum trial value of y. For example, in Fig. 12.13 the maxi- 
mum trial value y2 is located at z2. The true maximum ymax therefore 


Figure 12.14 
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must be contained within the interval 7; < x* < 2, to the left of 22, 
ort, < «* < a, tothe right of z.. Thus the total interval of uncertainty 
is 11 < 2* <a. However, if the values y; and y2 are not known, and 
if only locations x; and 22 are established, the interval of uncertainty 
will be either 0 < x* < 22 (presuming yi > yz) or 41 < x* <a (pre- 
suming y2 > yi). Since it can be reasoned that in the comparison of 
various search plans, or the examination of the convergence of a particular 
plan, it is not the location of the interval but rather the magnitude of 
the interval that is important, a conservative assumption, suitable for 
evaluating the effectiveness of a particular search plan, is to select the 
greatest of the possible intervals of uncertainty. (This is not to say that 
when a particular search is carried out the wrong segment should be 
retained for further subdivision. The procedure of defining the worst 
possible case is selected only to allow a conservative, quantitative 
evaluation to be made of the search plan itself, one that is independent 
of the particular variations in the dependent function.) 

In Fig. 12.12 it was demonstrated that the selection of only one trial 
value, regardless of where that trial might be located, will not allow a 
reduction in the interval of uncertainty. Two trials, however, will 
effect a reduction. The problem, then, is the determination of where 
these two trials should be placed. It is presumed that the range of vari- 
able of interest is a < x < b, where b — a = L (see Fig. 12.15). The 
segment in question is shown enlarged in Vig. 12.16. For the case 


Figure 12.16 
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shown, two possible intervals of uncertainty exist: 


1. If yi > ye, the intervalis {L,}-= ¢, + fo. (12.169) 
2. If yi < yo, the interval is {L2} = f, + 43. (12.170) 


The constraining equations for this elementary minimization problem, 


{Li} = 41 + 4 = minimum Pay 
{Lo} = 2 + 43 = minimum (12.172) 
are fi + f& + 3 = L Giza) 
and 4, >0 f. >0 £3 >0 (12.174) 


Solving Eq. (12.173) for f and substituting the resulting value into 
Eqs. (12.171) and (12.172) yields 


L — ¢; = mininum (12.175) 
and LL — 4 = minimum (12.176) 
A direct formal solution of these equations suggests that 
£, = 4; = 6L 
C277) 
with ty, = 


But the equation ¢; = 0 does not satisfy the second of the constraining 
equations (12.174). That is, the solution defined by Eqs. (12.177) 
requires that both trials be run at the same value of x. This, in turn, 
requires that the interval of uncertainty be the full value {1} and not 
{L1} or {Le} as presumed in Kas. (12.169) and (12.170). The correct 
placement of the two trials for {Zi} = {22} = minimum is shown in 
Fig. 12.17, where ¢ is any small positive value. 

Since a paired set of experiments has been carried out as indicated in 
Fig. 12.17, and it has been established that the optimum in question 
exists to the right (or to the left) of the centerline of the total region, a 


Figure 12.17 
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second set of paired trials can now be run about the three-quarter point 
(or one-quarter point). Paired trials about an eighth point would next 
be performed, and so forth. An indication of the effectiveness of this 
paired-trial method, or dichotomous search, as it is frequently called, 
is given in Table 12.3. 


Table 12.3 


Number of Trials Interval of Uncertainty 


L 
0.500L 
0.250L 
0.1252 
0.0625L 
0.0313L 
0.0156L 


NOAODrRN © 


— 


12.5.2 The Golden Section 


In the paired-trial search procedure just described, optimization was 
required at each and every stage. Such an assumption, however, does 
not necessarily guarantee the greatest search efficiency. A nonoptimal 
placement of “early” trial values may well result in greater efficiencies 
at later stages. A search pattern based on the golden section (or the 
golden ratiof) is one such method that will provide a greater search 
efficiency than the dichotomous search. The principle of the golden 
section can be stated as follows: 


In the division of a line into two unequal parts, the ratio of the larger of 
the two segments to the total length of the line should be the same as the 
ratio of the smaller to the larger segment. 


Consider the case shown in Fig. 12.18. The total length of the line in 
question is f. The unequal segments are a and b, where a > b. 
From the definition of the golden section, 


oo le 12.178 

ai ba ie ) 
where 7 is a constant. Since 

atb=¢€ (12.179) 


{ The golden section or golden ratio was known to Greek mathematician Euclid, fl.ca. 
300 s.c. It has had a profound influence on art and architecture, because of its con- 
nection with esthetics and visual beauty. 
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pate 


Figure 12.18 
substitution of the relationships given in Eq. (12.178) yields 
m+ra =7l+7(7f) = 0 
or (12.180) 
rP+tsee—l=0 
The single positive root of this equation is 
+ = +0.618 (12.181) 


The proportions of the golden section therefore are those shown in Fig. 
12.19. 

To illustrate the procedure to be followed in a golden-section search, 
consider the sequence illustrated in Fig. 12.20. The initial interval is 
presumed to be of unit length, as shown in Fig. 12.20a. The first trial 
would be placed as in Fig. 12.206, at a distance of 0.618¢ from the end. 
This single experiment, however, does not reduce the interval of uncer- 
tainty, as has been indicated earlier in this section. 

Since it is impossible at this stage to know whether the optimum in 
question lies to the left or to the right of the trial x = 0.618, the second 
trial (see Fig. 12.20c) should be so placed that the intervals of uncer- 
tainty are the same, {0.618}. If it is assumed that y(0.618) > y(0.382), 
the interval for further study is contained within the limits 0.382 < 2* < 
1.000. As shown in Fig. 12.20d, to afford equal intervals of uncertainty, 
the third experiment would be symmetrically placed at x = 0.764. Ina 
similar fashion the fourth trial would be at « = 0.854 (presuming the 
optimum lies to the right-hand side of the segment in question). For 


Figure 12.19 
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the fifth trial, x = 0.910. For the sixth, z = 0.944. Table 12.4 sum- 
marizes the intervals of uncertainty corresponding to the number of 
golden-section trials. These should be compared with those listed in 
Table 12.3 for the paired-trial search. 


12.5.3 Fibonacci Search 


It is to be noted that while the golden-section search described above is 
demonstrably more efficient than the dichotomous (paired-trial) plan for 
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Table 12.4 

Number of Trials Interval of Uncertainty 
0 L 
1 L 
2 0.6182 
3 0.382L 
4 0.236L 
5 0.146L 
6 0.090L 
au 0.056L 
8 0.0345L 
9 0.0213L 
10 0.0132L 
iil 0.0081L 
12 0.0050L 


n > 2, no optimizing, as such, was employed to obtain a “best”’ possible 
interval of uncertainty. The Fibonacci scheme with which this dis- 
cussion is concerned assumes a desired optimum after 7 trials. 

In developing this search plan it must first be recognized that sym- 
metrical placement of trials within a given segment results in greatest 
reduction in subsequent intervals of uncertainty. (Note, for example, 
Fig. 12.20.) Moreover, optimum placement of the last two experiments 
requires that they be an e distance one from the other, symmetrically 
placed about the centerline of the next-to-the-last interval. A search 
pattern can therefore be deduced by reasoning backward from these last 
experiments, as illustrated in Fig. 12.21. In all cases the trial point in 
question is shown as a solid circle. The interval of uncertainty is desig- 
nated by the brackets; for example, {L,} is the interval of uncertainty 
after the nth trial. Again, it should be understood that while for illus- 
trative purposes it has been presumed that the optimum occurs near the 
right-hand end of the original segment, the same reasoning and results 
would follow from any other assumption. 

Comparing the lengths of the intervals shown in Fig. 12.21a, b, and c¢ 
reveals a relationship between any three consecutive intervals of uncer- 
tainty. For the case illustrated 


{Ln-2} = {Ena} + {Ln} (12.182) 
For the more general case where 1 <2 <n 
{Li-2} = (ley {L;} (12.183) 


Again, working backward from the last trial (note the geometry of Fig. 
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12.214), 
{In-1} = 2{L,} — (12.184) 
Substitution of this into Eq. (12.182) yields 
tlino} = [200a} =e] aiden) = 3 (12.185) 


The interval {Z,_3} would be defined as follows: 
(Tis) = ey 


= 5{L,} — 2e (12.186) 
In similar fashion 
{Lins} = 8{ Ln} — 3e 
and (dg) = 13, } oe 
and so forth. These equations can be written in more compact form in 
terms of so-called Fibonacci numbers, where 


(12.187) 


{Dna} aa Fepaalny = gene (12.188) 
and Fo = 1 
Fy =1 (12.189) 


FP, = Fy Fy» Ro = tl 


The first 12 Fibonacci numbers are listed in Table 12.5. 

If it is assumed that the initial length to be investigated is 1.00, the 
interval of uncertainty after n trials can be determined from Eq. (12.188). 
Fe 

F, 


ee z Sear (12.190) 
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Table 12.5 
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The first experiment should be located at 


| Ors € 
r = 1 n : 
Seed giro (ir) F, (12.191) 


Table 12.6 lists the interval of uncertainty associated with the Fibonacci 
method as a function of the number of trials to be carried out. 


Table 12.6 
Number of Trials Interval of Uncertainty 
0 L 
1 L 
2 0.500L 
3 0.333L 
4 0.200L 
5 0.1252 
6 0.077L 
uf 0.0481 
8 0.02941 
9 0.01821 
10 0.01121 
it 0.0069L 
12 0.00431 
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12.6 Multivariable Search 


All the single-variable search methods developed in Sec. 12.5 presume a 
unimodal function. While such an assumption may be a reasonable one 
for many single-variable engineering-type problems, it is much less 
likely to be the case when the function in question is dependent on more 
than one variable. The value of a direct extension of the methods and 
reasoning just described is therefore questionable. 

Of even greater concern is the fact that optimization along a line—any 
line—in an n-dimensional space does not ensure that the value so obtained 
is an optimum for the total space, even though the function in question 
behaves in a unimodal manner along the line being examined. This 
fact suggests that for multidimensional problems, instead of behaviors 
along lines being considered, regions within the total space be examined. 
A direct comparison of the optimum function values within each region 
could then be used to determine that a particular region be excluded from 
further consideration, or that it exclude the comparison region. Such 
methods have been developed. However, they will not be discussed 
here. Instead, a more direct continuous procedure will be developed. 


12.6.1 The One-at-a-time, or Sectioning, Method 


A logical extension of the one-dimensional case discussed in See. 12.5 
would be to select a starting point—any starting point—for the search 
and, holding all but one of the independent variables constant, alter 
that one until an optimum is realized. Then, in a manner not unlike 
that of the Gauss-Seidel iterative method for solving sets of algebraic 
equations, holding constant the just-determined independent variable 
and all others but one at their previously assigned values, allow a second 
variable to seek its relative optimal value. In turn, by altering only one 
variable at a time, holding all others constant, all independent quantities 
are allowed to vary. The procedure is continued until no future improve- 
ment is realized. This is illustrated for a two-independent-variable 
problem in Fig. 12.22, where f = f(x1,42). In the figure, contours of 
equal values of f are shown. The particular search plan followed is 
shown by arrows. ; 

The method just described is known as sectional search or the one- 
at-a-tume method. When the function in question is convex, and when it is 
symmetrical about,a pair of axes parallel to the coordinate axes, it is an 
extremely efficient search technique. This is illustrated in Fig. 12.23a. 
The case where the axes are at angles to the coordinate directions is shown 
in Vig. 12.236. Here, the method results in the correct solutions being 
found, but more steps are involved. 


For certain types of functions, and certain starting locations for the 
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Contours of equal value 
of F(x4,X2) 


D) 


xp 4 


i Start 
= 


x 


Figure 12.22 


search, the one-at-a-time method can yield completely false locations for 
the optimum solution. One such case is illustrated in Fig. 12.24. There- 
fore, unless more information concerning the general behavior of the 
function is known in advance, sectional search is not recommended. 


12.6.2 The Method of Steepest Descent (or Ascent) 


Also known as the method of gradients, the method of steepest descent has 
found wide acceptance as a powerful multidimensional seek procedure. 
It can be used for both nonconstrained and constrained types of problems, 
as well as for the solution of extremum situations. 
Consider first the problem of defining in an open domain the optimum 
of a function 
i — a ee ae) (12.192) 


It is presumed that the function possesses continuous partial derivatives 
with respect to each of the independent variables, and that the minimum 
value of f is desired. 


Figure 12.23 
Xo \ x2 K 


S 
Start e 
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Xok 
Figure 12.24 
From a starting point #1, 2, . . . , &n, a ds change is presumed, where 
(ds)? =_(da)? + ae)? 4+ * ~ = =— (da,)7 
(dx? 

1— —}) =0 12.193 
or y (=) ( ) 
This change, however, alters the value of the function f. 

0 fe) fe) 
= Mik gis + oe Se oo ui dia (12.194) 
OX OX» Ol, 


With respect to the total coordinate relocation ds, the rate of change in 
the function value is 


af afde , fdr | | af dts 
ds 0x1 ds Ox, ds OXn ds 
dfs <<. Of de: 
dt oe 12.195 
* ds ” dx; ds ( ) 


The (dz,;/ds)’s are direction cosines associated with the variation. 

In formal terms, the problem at issue is the minimization of a function 
[Eq. (12.195)], subject to a constraining equation (12.193). This can 
be accomplished by using the lagrangian-multiplier method discussed in 
Sec. 12.2. The augmented function is 


Of dx; | (dai? 
et 12.1 
oy dx; ds y esl (F) | ( Be) 
where ) is the constant lagrangian multiplier. (It is to be recognized 
that df/dx; in Eq. (12.196) can be presumed known.) The unknowns 
are the n particular direction cosines dx;/ds, which result in the greatest 


rate of reduction (for steepest descent) in the function value, and the 
lagrangian multiplier. 
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Differentiating each of Eqs. (12.196) with respect to dz,;/ds and equat- 
ing the resulting expression to zero yields 


of dx; , 
+1(-2&) <0 Lee ae en (12.197) 


OX; Ss 


These, together with the constraining equation (12.193), allow solution 
of the unknown direction cosines and the lagrangian multiplier. 


dey uy of = af \2]—* 
ds a 0x4 BS ea 


=1 


dx» of “ of 21) —-2% 
SS ee = 2 
ds ~ 0X2 Be a | le 
din % aly Cis 
ds — OhnL , \Ox: 
a of \? 16 
and A= + Al (+) | (12.199) 
fay \O%: 


With these equations, the rate of change in the function value can be 
determined by substitution in Eq. (12.195). 


ae  / of \? |” 
PG) a a 


t=1 


(In all these equations, the plus values following the equals signs cor- 
respond to steepest ascent, whereas negative values are for steepest 
descent.) 

Consider now the introduction of a time parameter ¢, and presume that 
motion along the direction of steepest descent vs a continuous process of treme. 


Since 
ds — (dx? 1” 
ane Oe 
meta) (12.201) 
dx; dz, ds 
25 = 12.202 
ine dt ds dt ( ) 
dx; _ Of ds— o / af\?|-% 
dt a; dt » (4 | ee 


If it is now presumed that the magnitude of ds/dt is proportional to 
df/ds, the expressions become relatively simple. 
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ds cr on i 
es sets 12.204 
as » @ (12.204) 
dx a) 
oe 2 Kee 
ane dt (x 
dx. of 
hater AS PI 76 
dt (<7) (12.205) 


The point (21,%2, . . . ,&n) moves according to this system of equations. 

As an alternative to the continuous procedure just described, a step- 
wise approach, one that corrects a set of approximations to the solution 
of/dx; = 0 by increments proportional to the negative of df/dzx;, can be 
defined. From Eqs. (12.205), 


dx; of : 
= hi = [25 ee, 12.206 
R= -K(Z) =42-..,9 042.206) 
For the discontinuous process, this would correspond to 
0 
Av; = —K (#) At (12.207) 
Ox; 


By noting ,v, as the value of the coordinate 2x; corresponding to the 
pth step in the iterative process, Eq. (12.207) can be written in the more 
usable form 


0 
p41ty. = ote ( “) (K At) tle, Ree a OS) 
D OX; 


Example 12.9 Determine the minimum value of the function 
f (1,22) = Obie + oa (12.209) 
presuming the starting value x; = 12, zr, = 6. Use both the continuous 


and the stepwise approach. 


ConTINUOUS SOLUTION 
From Eqs. (12.205), presuming K = 1, 


iad eee Nor 
dt a O24 '; 7 
(12.210) 
and aes =— oh = —22, 


dt OX» 
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These, plus the given starting value (12,6), yield the solutions 
eee 12e-% Lo = be-*# (12-21) 


Since ast > ©, both x, and x; > 0, where df/dx, = df/dx2 = 0, the mini- 
mum value of the function corresponds to 


lb Se f(x1,22) = (0) 22212) 


STEPWISE APPROACH 
For illustration, it will be presumed that K = 1.0 and At = 149. The 


solution for the first 10 trials is shown in Table 12.7. Thé iterative 
function is 


: hs 7 (127213) 
ay ae Mri ae i= 
tee yet 10 ,\0z; : 
Table 12.7 
Ie pli => p+it1 pve ay p+102 
D 02 Pp OX» 
1 12 d2 3.8 6 12 4.8 
2 5.8 34.8 2232 4.8 9.6 3.84 
3 2.32 13.92 0.928 3.84 7.68 3.072 
4 0.928 5.568 0.371 3.072 6.144 2.458 
5 0.371 22226 0.148 2.458 4.916 1.966 
6 0.148 0.888 0.059 1.966 3.932 ia} 
7 0.059 0.354 0.024 iL ave 3.146 1.258 
8 0.024 0.144 0.010 1) DABS} 2.916 1.006 
9 0.010 0.060 0.004 1.006 2.012 0.805 
10 0.004 0.024 0.002 0.805 1.610 0.644 
The path of convergence is shown in Fig. 12.25. 
Figure 12.25 


Starting point, p=1 
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Consider next that particular case where a function 


hi = i Caos bea & Oi (12.214) 
is to be optimized, subject to the given set of constraining equations 
OilCiybo, 6 een Ba Nk Ws ce lng (12,215) 


It is to be recognized that the additional geometrical constraint [see 
Eq. (12.193)] 


GEN 
= = 0 ‘ 
1 me (=) (12.216) 


must also be included. To facilitate inclusion in the optimization, Eqs. 
(12.215) are normally expressed in terms of the rate of change with respect 
to the distance ds. That is, 


n 


dy; Og; dx; 
) OX; ds 


ee ea) (12-217) 
ds 


=0 j=l 


j 


i=1 


The augmented function to be optimized contains m + 1 lagrangian 
multipliers. 


ee ro[ 1 — a | yy eee ao. 
ae ee 2 ds ue ; 2 dx; ds eels) 


=1 


Differentiating, partially, each of these equations with respect to (dx;/ds) 
and equating the resulting expression to zero yield 


of ‘dx; 2“ Og; 
iN et ee ee soeel ee ae se 
Ox; : (=) 7 » Z (3) v if t} ) nN (12: 19) 


g=1 


These, together with the m+ 1 constraining equations (12.216) and 
(12.217), allow solution for the n direction cosines, \o and the m multi- 


ONS newer Sa 5 Sey 
athe \ Of de;\ |?|” 
east 2 lant 2 GE)|P zm 
de. (Of /dx;) SF x, hj (0¢;/02;) 
ag. eee = (12.221) 
~ {1 ), [@f/dx) + Y s(@ei/an,) ]?)* 
t=1 j=l 


Substitution of Eqs. (12.221) into Eqs. (12.217) allows determination of 


Mi, Ao,» - » » Am. The m X m set of equations to be solved is shown in 
Table 12.8. 
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Again, as in the preceding case, a continuous time parameter ¢ will be 
presumed to control “motion” along the direction of steepest descent. 
[See Eqs. (12.201) to (12.203).] By taking ds/dt proportional to the 
denominator of Eq. (12.221), the direction cosines are obtained as 


dx, of Og Oye dg 
Se at peat’ mee ae pect es 
dt BE sled G2) Bue? 62) a thm G=)| 


dx of (2) 62) Om 

— = —k} — N eae » oes 55 6 m(—— 

dt es oe OX» om OXs ss oe: (| (12.222) 
din Pe of 0g, Ove OGm 

ca ke a eS ee ce) os ae Cl 


It is to be noted that the d’s indicated in (12.222) are to be obtained from 
solution of the equation given in Table 12.8. 


A stepwise approach for steepest descent would require an iterative 
solution of Eq. (12.223). 


ye Io dy; 
ptiti = pli — i rie a dj (6) | (K At) Gen 28 sew yD 
(12.223) 


Here again, the \,’s must be obtained from the equations in Table 12.8. 


Example 12.10 For the structure and loading shown in Fig. 12.26, 
determine the cross-sectional areas corresponding to minimum weight 
of total truss, when the deflection in the direction of the applied load is 
constrained to equal 6. (See Example 12.1.) 

With the notation of Example 12.1, the objective function (weight) 
to be optimized is { 

W = C1, (V2.4, + A) 


or f(t1,t2) = V2 4, + 2 (12.224) 
where ty Ais and Le = Ae (12.225) 
The deflection constraint equation is 

ava i 
AiC, AxC2 


9) 
Vergo (9S 5 (12.2266) 
Hos ve 


Btls Ay) = 1=0 (12.226a) 


or g(x1,22) = 


a 


ure Yee xe %xe Txe txre “re “Ze exe %*xe Ize xe 
Tn SESS - i a Uu. Us, = on [“N] UL, uw aie aa? ual a Uu UL a9 UL UL, als ae oF 
fe “de fe “te fe “de de “de de “dhe ae “de 
“ZO “LO exe %*xe Tze 'xe CHO) SH) exe %xe Tze 'xe 
[PN] (ee ee et + SmaI eee! «ee ae ar 
e “de we tAp tae “Ae Tae “de Tae “dp the “de 


aq pynom asayz fo uounnba yyw ay, ‘“suounnba buruv.ysuoo fo suaqunu abun) sof uazum aq pjnoa suorwnba Tigo spjvury 


“Ze “XO Sze %xe Tze Txe 4 “xe “xe °Ze *xe 're 'xe “xe “Ze %xe %xre ‘ze 'xe 
SS an a Se SS See oy eee ee od 6 ae =e f+... ~ ———— +4 —_ __ 
fe to fe fa7e) fe ze tAe the tA the the %de Ide td0e TAQ tA0 TAG tAO 
“to “Te exe %xe Tze 'xe a uve UTE exe *xrE Tze 'xe uZe “Ze exe *xp Tze 'xe 
a ie = rare Ty tee oe oe eee ren aS ey ee ee 
fe 'e fe te fe ‘de e 14@ the ttp the Ide }\ Ide Ihe The 1t9 de Ide 
I Or (ee Sr Ge 60 “x1 ea F 
QO = (“a + + + &xIx)Id :suoyonba bururp.ssu0d on} LO 
yo ayo exe %xre Ize 'xe “Te: *xLeE exe xe Tze 'xe 
ees ior oie =e fet = [N][- de og o alle 
fe '4e JOM Pe Te 6 eo 126 loc oes Sice20 
0 = (“a + + + &xIx)Id :uounnba bururp.ijsuos suo 40g 
SGT 2199.1 


474 


optimization 475 


Figure 12.26 
The various partial derivatives required in the solution are 

Op _ 2+/2 dg il 
0X1 Tae OX _ oe 

2 : (12.227) 
0 = 0 
— = 4/2 SS 
Ox; Vv OX. 


Since there is only one specified constraining equation, there is only 
one lagrangian multiplier to be determined. Using the first equation 
of Table 12.8, 


(e+ 
(2) w-(-3)0 


Sey WG tea 
or ay! ato! Da ar? 02 
4 + a? F 
= 12.228 
or Al ae ( ) 
je 
where a=— (12.229) 
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By presuming a stepwise approach, the equations for iteration are 


ppiti = pli — | v2 1 (= : v=) (K At) 


D x 


(12.230) 
1 
pt1t2 = ple. — E “fb u (= =) (K At) 
p i) 
Substituting in the value of \1 from Eq. (12.228), 
aS 4+ a 
pt1t1 — pti aad 4/2 (1 ines 2 8 =) (K At) 
: (12.231) 
; 4 + a 
pie? — plo = (1 — a 84 =) (K At) 


Starting values for the seek routine must satisfy the given constraining 
equation (12.226b). For example the following values are chosen: 


ity = 44/2 
12 = 2 


(12.232) 


Table 12.9 summarizes the first five steps of the solution. It is to be 
noted that (K At) has been arbitrarily selected equal to 1. 


Table 12.9 
v1 
iP mL pt a=— K At EGER ptit2 
2 
0 5.656 2.000 
1 5.656 2.000 2.828 1.000 4.713 2.333 
2 4.712 PY RS} 2.020 4.225 2.671 
3 4.240 2.673 1.586 4.112 281% 
4 4.113 2.815 1.461 4.081 2.858 
5 4.079 2.860 1.426 4.071 2.871 


One of the major difficulties in using the method of steepest descent 
is the selection of the magnitude of K At. The smaller the value, the 
more closely the stepwise approximation will approach the continuous 
solution. As the number of constraints increases, however, the deter- 
mination of the lagrangian multipliers, and thereby the gradient direction, 
becomes more involved—at an increasing rate. Therefore the exploita- 
tion of each step has been suggested as a desirable approach to use in the 
solution of these types of problems. That is, at each trial point Pare 
the direction of steepest descent is determined by using the values at 
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that point. ds variation in that steepest-descent direction is then con- 
tinuously increased until a rate of change of zero is noted in the objective 
function. At that new location, a new gradient direction is determined, 
and the entire process is repeated. Such a procedure is similar to the 
one-at-a-time method described earlier, with the exception that the 
directions are now gradient directions, and are not necessarily parallel 
to the coordinates. 

Gradient methods have an inherent advantage: They avoid saddle 
points. This is illustrated for the two-dimensional case in Fig. 12.27. 
Three typical paths are shown. Path @ follows the valley to the saddle 
point, and then proceeds to move to either () or @). Paths @) and (3) 
proceed directly to @ or @). 

Figure 12.27 points up one of the major difficulties associated with the 
determination of absolute maximum or minimum values within a given 
region. If more than one relative maximum or minimum exists, a seek 
routine will “lock in” on one location, and not find any of the others. 
An obvious approach toward a solution to this troublesome situation 
is to select a variety of widely scattered starting points. If enough paths 
are examined, and if all result in the same location for the optimum, it 
may be reasonably concluded that the solution found is the absolute 
optimum within the region in question. 

It is to be recognized that the procedures defined in this section can be 
used also to experimentally solve “optimum problems,” even when the 
functions (or constraints) in question are undefined. At each trial loca- 
tion a sufficient number of closely spaced tests would be carried out. 
Normally, a minimum of n + 1 tests is required, when n independent 
variables are present. From these, a direction of steepest descent can 
be ascertained. This is shown diagrammatically in Fig. 12.28 for the 
case of two independent variables. 

Finally, it must be understood that the methods hereinbefore developed 
presume a euclidean type of geometry. That is, for a given problem the 


Figure 12.27 


Xo h Path) 
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xoh 


Test pointso 


Starting point 


Figure 12.28 


coordinate functions—the independent variables—must be of the same 
dimensions. “Optimum problems’’ in engineering and applied science, 
however, frequently contain mixed coordinate variables: length, force, 
temperature, pressure, cost, etc. To be able to use these methods for 
each situation, then, it is necessary to nondimensionalize all coordinate 
quantities. That is not to say that all are nondimensionalized by the 
same quantities, but that each is related to its appropriate scale factor. 


Prob. 12.1: Determine the minimum value of the function 
f =a? + 2x, + 8x? + 4(1 — a) 


Prob. 12.2: In the range 0 < 2, < 1.0 and 0 < x < 1.0, determine 
the maximum and minimum values of the function 


Ue ae + U1X2 + Saor 
subject to the constraining equation 
v1 — MWe = 0 


Prob. 12:32 In¥ the range .O;= a 1) 0) Se 0 
0 < a, < 4, determine the minimum value of the function 


yY = 3%, + 22. + 23 + w,? 
subject to the following constraining equations: 
Li + 2x42. = 1 Li + wa = 0 
Prob. 12.4: Vind the minimum distance between the two curves 


y=2?—272+2 and y=1-2 
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Prob. 12.5: For the three-bar pin-connected indeterminate truss 
shown in Fig. 12.29, the horizontal component of the deflection at the 
load point is given by the equation 


E AiA, + +/2A1A;3 + A2A; 


The cross-sectional areas of the various members are denoted as Aj, 
As, and A3. The same material is used throughout the structure, and 
has a Young’s modulus of elasticity of H. 

It is desired to obtain for a fixed value of 6 the relationships between the 
cross-sectional areas which correspond to the least total weight of the 
members of the truss. 

In more formal terms, the problem is to determine the minimum value 
of the linear function ¢, subjected to the given nonlinear constraint 6. 


g = V 2x1 + a2 + 0/223 


QL? 2114+ 2+/22.+ 23 
E ate + V/2air3 + 2023 


= 


where 21; = A,L 


Prob. 12.6: In a two-dimensional space, prove that a straight line is 
the shortest distance between two points. 


Prob. 12.7: Given the two points A(1,0,0) and B(0,1,1) on the 
cylindrical surface whose equation is 


ety? =1 
determine the equation of the curve which corresponds to the shortest 


distance between these two points (see Tig. 12.30). 


Prob. 12.8: A column of uniform cross section and length L, pin- 
connected at one of its ends and rotationally restrained at its other end, 


Figure 12.29 
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i 


Figure 12.30 


is subjected to axial thrusts as shown in Fig. 12.31. The potential energy 
of the system is given by the equation 


LfEI , ie , B fj 
U= ff (Fv —Sut)act Sue 


where 8 = restraining spring constant at x = 0 
yy = slope of member at x = 0 
The boundary conditions are 


y(0) =O = EIy’’(0)] + By’(0) = 0 
yl) =O y'"(L) =0 


Since equilibrium requires that the potential energy of the system be a 
minimum, determine the value of the critical axial thrust corresponding 


Figure 12.31 
i 


oe tA 
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to the first possibility of lateral deformation, i.e., corresponding to a 
minimum value of U, by each of the following methods: 

(a) Using the appropriate Euler equation, and solving the resulting 
differential equation 

(b) Using Ritz’s method, assuming the smallest-order polynomial that 
the method and boundary conditions will allow 

(c) Using Galerkin’s method, assuming the smallest-order polynomial 
that the method and boundary conditions will allow 

Prob. 12.9: In the range 1 < 2 < 3, 1 < x S05, 0) a 10a 
function y depends on three independent variables 21, 2, and x3. The 
following table lists measured values of y at nine different locations. 


Point zy Xe 2X3 i] 
1 =F il sF il se +4.0 
2 +2 seh +8 +3.0 
3 +3 +1 dL +25 
be SP +2 +8 +5.0 
5 +2 +2 +6 +2.3 
6 +3 +2 +4 +3.0 
qf sl +3 sei +8.0 
8 +2 +3 +4 +4.0 
9 +3 +3 +1 +5.0 


Starting from location x, = 2, z, = 1, and z3 = 8 (that is, point 2), 
using the method of steepest descent, determine the location and value 
of the minimum y. 


poe O16 Oe S 


_ 
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Absolute error, 99 
Absolute maxima, 429 
Accuracy, analog computer, 7 
Adams-Bashford method for analytical 
continuation, 283 
Addition of matrices, 45, 63 
Algebraic equations, 66-117, 158-187 
linear, 3 
nonlinear, 3 
Analog computer, 5-7 
Analytical continuation, 251, 280-286 
ordinary differential equations: col- 
location method, 285 
predictor-corrector methods, 
281-285 
Arc length, minimum distance, 447 
Array: 
multidimensional, 20 
one-dimensional, 14 
two-dimensional, 14 
Augmented groups: 
equations, 437, 444 
matrices, 77 


Back substitution, 72 
Backward differences, 199, 228 


Banachiewicz method (square-root 
method), 81 
Beam-column bending problem, 52, 
440 
Bessel’s integration formula, 232 
Bessel’s interpolation formula, 210 
Biharmonic function: 
equation, 320, 348 
operator, 320 
Boundary conditions: 
closed (boundary-value problem), 2 
geometrical, 450 
irregular, 335 
open (initial-and-boundary-value 
problem), 2 
Boundary-value problems, 246, 287— 
296 
elliptic partial differential equations, 
343-379 
nonlinear, 301-314 
relaxation methods (elliptic partial 
differential equations), 359-379 
Branching, 16 
Buckling: 
flagpole column, 142 


plate, 352 
485 
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Calculus of variation, 438-455 
Central differences, 227 
Characteristic direction: 
hyperbolic differential equations, 403 
nonlinear equations, 416 
Characteristic polynomial equation, 
138 
Characteristic-value problems, 296-301 
(See also Eigenvalues) 
Cholesky’s method: 
flow diagram, 115 
solution of inverse problem, 123-127 
solution of linear algebraic equa- 
tions, 76-81 
Closed boundary, 319 
Closed integration formulas, 281 
Closed region, 319 
Cofactors, 25 
reduced, 120 
Collatz, L., 455 
Collocation method, 256, 285 
analytical continuation, 285 
Column buckling, 142 
Column vector, 41 
Compilers, 8 
Computational molecules, 323-335 
laplacian: regular grid, 319, 349, 367 
unequal grids, 376 
polar coordinates, 335 
skewed grid, 333 
square grid differential operators, 
327 
higher-order, 328 
volume, 329 
triangular-coordinates, 333, 336 
Computers: 
analog, 5-7 
direct, 7 
indirect (mathematical), 7 
digital, 5 
hybrid, 5 
programming, 7—22 
analog, 7 ; 
compilers, 8 
conversational, 8 
destructive read-in, 19 
digital, 5 
Fortran, Mad, and Algol, 8, 19 


Computers: 
programming: machine-language 
coding, 8 
problem-oriented programming 
languages, 8 
Constrained maximum (minimum), 
431 
Constraining equations, 434 
Contour maps, 161, 429 
Control theory, 432 
Control variables, 427 
Convergence: 
iteration for roots of nonlinear 
equations, 174, 182-184 
parabolic partial differential equa- 
tions, 385 
Convexity, 429 
Coordinate transformation, 42, 59 
Cramer’s rule: 
eigenvalue problems, 137 
solution of inverse problem, 119-122 
solution of linear algebraic equations, 
69-71 
Crank-Nicolson formulation: 
hyperbolic partial differential equa- 
tions, 421 
parabolic partial differential equa- 
tions, 397, 400 
Crout’s method (see Cholesky’s 
method) 
Cumulative multiplication, 172 
Curve fitting using least-squares 
method, 233-238 


d’Alembert forces, 54, 141 
De network, 55 
Del, 319 
Descartes’s rule of signs, 164 
Destructive read-in, 19 
Determinants, 24-39 
cofactor, 25 
methods of evaluation: expansion 
by minors, 25 
pivotal condensation, 29 
row or column addition, 28 
minor, 25 
pivot, 28 
transpose, 27 


Diagonal matrices, 43 
Dichotomous search, 460 
Difference operators: 
backward, 199 
central, 201 
A, V, and 6, 197-204 
forward, 198 
interrelationships, table, 202 
shift operator, 198 
Difference tables, 188-197, 241-243 
divided, 194 
errors, 195-197 
Differences, 190, 222 
backward, 199, 228 
central, 227 
forward, 198, 226, 241 
higher-order, 203 
“off-center” forward, 289 
Differential equations, 245-426 
eulerian, 440 
table, 443 
least-squares approximation, 257 
Differentiation, 220-229 
tables of equations, 226-228 
Digital computer, 5-23 
fixed-point numbers, 14 
floating-point numbers, 14 
flow diagrams, 9 
Divided-difference table, 194 
Domain, 319 


Eigenvalues, 136-157 
flow diagram, computer solution, 155 
ordinary differential equations, 296— 
301 
partial differential equations, 352, 
355 
solution by iteration, 145-147 
largest value, 147 
smallest value, 147 
Eigenvector, 138 
Elementary row operators, 122 
Elimination methods, 71-83 
back substitution, 72 
Cholesky’s method, 76-81 
Crout’s method, 76 
Gauss’ method, 71-76 
inverse problem, 123-127 
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Elliptic partial differential equations, 
319, 3438-379 
Equations: 
augmented, 437 
characteristic polynomial, 138, 139 
constraining, 434 
contour maps; for nonlinear equa- 
tions, 161, 429 
differential, 245-426 
homogeneous, 136, 246 
Lagrangian form, 233 
linear, 66, 245 
nonlinear, 66, 158-187, 245 
secular, 138 
transcendental, 175 
Errors, 3 
absolute, 3 
approximate methods of solution, 3 
difference tables, 195-197 
double or multiprecision solutions, 3 
inaccuracies in initial data, 3 
interpolation formulas, 217 
lack of precision, roundoff, 3 
propagation, 196 
relative error, 3 
rounding off, 217 
truncation error, 3, 217, 221, 287 
Kuclid, 460 
Euler-Cauchy method, 263 
table, 274 
Eulerian differential equation, 440 
table, 443 
Euler’s method, 262-267 
table, 274 
Everett’s integration formula, 232 
Everett’s interpolation formula, 209 
Existence of a solution, 2 
Explicit recurrence: 
hyperbolic partial differential 
equations, 418-420 
parabolic partial differential equa- 
tions, 380-395 
Extremum, 488, 451 


False-position method, 168-170 
Fibonacci numbers, 464, 465 
Fibonacci search, 455, 462-465 
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Finite-difference method, 418 
(See also Collocation method) 
Fixed-point numbers, 14 
Floating-point numbers, 14 
Flow diagrams, 9 
branching, 16 
Cholesky’s method for obtaining 
the inverse, 127 
Cholesky’s method for obtaining 
solution of simultaneous linear 
algebraic equations, 115 
compute box, 9 
counter, 12 . 
eigenvalue determination, 15 
Gauss-Seidel, solution of simul- 
taneous linear algebraic equa- 
tions, 103 
looping, 13 
matrix addition, 63 
matrix multiplication, 65 
nesting, 14 
pivotal condensation, evaluation of 
a determinant, 38, 39 
question box, 10 
Forward differences, 198, 226, 241 
Functional, 438 


Galerkin’s methods: 
optimization, 452-455 
solution of ordinary differential 
equations, 258 
Gauss elimination method: 
solution of inverse problem, 126 
solution of linear algebraic equa- 
tions, 69-76 
Gauss-Seidel method for iteration, 85 
flow diagram, 103 
Gauss’ backward interpolation 
formula, 209 
Gauss’ forward interpolation formula, 
209 
Geometrical boundary conditions, 450 
Golden section, 455, 460-462 
Gradients, method of, 467-478 
Grids, 320-323 
change in size, 371-377 
Group relaxation, 92, 364-370 
Guest, J., 182 


Heun’s method, 273 
table, 274 
Homogeneous equations, 136, 246 
Hybrid computer, 5 
Hydraulic network, 57, 367, 377 
Hyperbolic partial differential equa- 
tions, 319, 401-421 
finite difference methods, 418-421 
method of characteristics, 401-418 


Identity matrix, 44 


Ill-conditioned systems, 68 


Implicit recurrence: 
Crank-Nicolson formulation, 397, 
400, 421 
hyperbolic partial differential equa- 
tions, 421 
parabolic partial differential equa- 
tions, 395-400 
Inaccuracies, 3 
(See also Errors) 
Influencing interval, 194 
Initial-value problems, 246 
hyperbolic partial differential equa- 
tions, 319, 401-421 
explicit recurrence, 418 
implicit recurrence, 421 
ordinary differential equations: 
Euler-Cauchy method, 263 
table, 274 
Euler’s method, 262-267 
table, 274 
higher-order equations, 259-261 
Huen’s method, 273 
table, 274 
Kutta-Simpson (one-third rule) 
method, table, 274 
Kutta-Simpson (three-eighths 
rule) method, table, 274 
Picard’s method, 253 
Runge-Kutta methods, 268-280, 287 
Taylor series solution, 247-252, 
259, 268 
parabolic partial differential equa- 
tions, 319, 379-401 
explicit recurrence, 380 
implicit recurrence, 395 
stability, 286, 381, 385 


Instability, 286, 381, 385 
Integration, 220, 229-233 
Bessel’s formula, 232 
closed, 281 
Everett’s formula, 232 
formulas, 234-236 
Lagrange formula, 229 
Newton’s forward difference formula, 
232 
Simpson’s one-third rule, 231, 244 
tables of formulas, 234-236 
trapezoidal rule, 230 
Interpolation, 204-220 
Bessel’s formula, 210 
Everett’s formula, 209 
Gauss’ backward formula, 209 
Gauss’ forward formula, 209 
inverse, 218 
Lagrange formula, 214, 215, 229, 243 
linear, 204 
Newton’s backward difference, 207, 
209 
Newton’s forward difference, 206, 
209 
table of formulas, 209-210 
Interval of influence, 194 
Inverse interpolation, 218 
Inverse problem, 118-135 
Cramer’s rule, 119-122 
elementary row operators, 122 
elimination methods, 123-127 
Cholesky’s method, 127 
Gauss’ elimination, 126 
partitioning, 128-131 
Irregular boundaries, 335-343 
Iteration methods, 83-87, 145, 253-255 
convergence, 174, 385 
“educated guess,” 83 
eigenvalues, 145-147 
false-position, 168 
Gauss-Seidel method, 85 
Jacobi’s method, 84 
nondiagonal sets, 87 
nonlinear equations, 174, 182-184 


Jacobian matrix, 309 
Jacobi’s method for iteration, 84 


index 489 


Kenneth, P., 304, 307 

Kirchhoff’s law, 55 

Kutta methods, 268-280, 287 

Kutta-Simpson method: 
one-third rule, 274 
three-eighths rule, 274 


Lagrange interpolation formulas, 214, 
215, 229, 243 
errors in, 217 
inverse interpolation, 218 
numerical integration, 229 
Lagrangian form of equations, 233 
Lagrangian multiplier, 432-438 
Language: 
conversational, 8 
Fortran, Mad, and Algol, 8, 19 
machine coding, 8 
problem-oriented, 8 
Laplace operator, 319 
change in grid size, 371 
computational molecule, 319, 349, 
367, 376 
Lattice search, 455 
Least-squares approximation: 
curve fitting, 233-238 
solution of differential equations, 257 
Leitmann, G., 431 
Linear algebraic equations, 66-117 
solution of: Cholesky’s method, 76- 
81 
Cramer’s rule, 69-71 
Gauss elimination, 71-76 
Gauss-Seidel iteration, 85 
Jacobi’s method, 84 
relaxation methods, 87-94 
square root method, 81-83 
Lines, method of, 400 
Local maxima, 429 
Looping in flow diagramming, 13 
Lower triangular matrix, 44 


McGill, R., 304, 307 
Matrix (matrices), 40-65 
augmented, 77 
column vector, 41 
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Matrix (matrices) : 
diagonal, 43 
general, 4, 40-65 
Jacobian, 309 
lower triangular, 44 
multiplication of: by a constant, 49 
by the identity matrix, 48 
null (zero), 44 
partitioning, 49-51 
bordering, 50 
naturally partitional, 50 
submatrices, 51 
row vector, 41 
square, 20, 43 
transpose, 21, 45 
unit (identity), 44 
upper triangular, 44 
upper unit triangular, 44 
Matrix addition, 45, 63 
Matrix multiplication, 46, 47, 65 
Maxima, 427 
absolute, 429 
constrained, 431 
local, 429 
Membrane, natural frequency of 
vibration, 355 
Milne, W. E., 282 
Minima, 427 
absolute, 429 
constrained, 431 
local, 429 
Minimum distance, 436 
Minor, 25 
Molecules (see Computational mole- 
cules) 
Multiplication of matrices, 46, 47, 
65 
Multiplier, Lagrangian, 432, 433 


Natural frequency of vibration: 
membrane, 355 
two-mass, spring-connected system, 
140 
Nesting in flow diagramming, 14 
Network: de, 55 
hydraulic, 57, 367, 377 
Newton-Raphson method, 304, 308 


Newton’s methods: 
backward difference interpolation, 
207 
errors in interpolation formulas, 217 
forward difference integration 
formulas, 232 
forward difference. interpolation, 206 
roots of nonlinear equations: first- 
order method, 170, 186 
second-order method, 171, 181 
Nonlinear equations, 66, 158-187 
Null matrix, 44 
Number sorting, 17 


Objective function, 436, 473 
“Off-center” differences, 289 
Open domain, 319 
Open integration formulas, 281 
Operator: 

biharmonic, 320 

elementary row, 122 

square grid, differentials, 327-329 
Optimization, 427-481 

Galerkin’s method, 452 
Ordinary differential equations, 245— 

317 
boundary-value problems, 246, 287, 
301-314 

eigenvalues, 296-301 

initial-value problems, 246 
Orthogonality method, 258 


Paired-trial search, 460 
Parabolic partial differential equations, 
319, 379-401 

Partial differential equations, 318-426 
eigenvalues, 352, 355 
elliptic equations, 319, 343 
hyperbolic equations, 319, 401-421 
parabolic equations, 319, 379-401 

Partitioning, 49 
inverse problem, 128-131 

Picard’s method, 253 

Pivotal condensation, 29 
flow diagram, 38 

Plate buckling, 352 


Poisson’s equation, 319, 346 
Polar coordinates, 335 
elliptic problem, 349 
Polynomial approximations, 255-259 
Precision, 3 
Predictor-corrector methods, 281-285 
Problem types: 
boundary-value, 2, 246, 287 
characteristic-value, 1, 296 
eigenvalue, 1 
initial-value, 2, 246 
inverse, 123 
steady-state, 1 
transient, 1 
Propagation of errors, 196 
Propagation problems, 319 
Proper values, 138 
Proportional parts, 204 


Quadrature formulas, 231 
(See also Integration, formulas) 
Quasi-linear partial differential equa- 
tions, 319 


Raphson-Newton method, 304, 308 
Recurrence: 

explicit, 380, 418 

implicit, 395, 397, 400, 421 
Reduced cofactor, inverse problem, 120 


Region: 
closed, 319 
open, 319 


Relative error, 3 

Relaxation methods, 87—94 
group-, 92, 364-370 
residuals, 89 
unit-, 89, 360-364 

Residuals, 89, 361 

Ritz’s method, 449-452 

Roots of equations: 
approximations: largest, 163 

smallest, 164 
conjugate complex, 162, 177 
nonlinear equations, 170, 171, 181, 
186 

real, 161, 180 
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Rope, deflection of, 446 
Round off, 3 
errors, 217 
Round off error (see Errors) 
Row operators, elementary, 122 
Row vector, 41 
Runge-Kutta methods: 
method of lines, 400 
ordinary differential equations, 268- 
280, 287 


Saddle point, 430 
Scale factors, 93, 478 
Scanning (nonlinear equations), 159 
Search methods, 455-478 
multidimensional problems, 466-478 
method of gradients, 467, 477 
one-at-a-time method, 456, 466 
pattern search, 456 
sectioning, 456, 466 
steepest ascent, 455, 467 
single-variable problems, 455-465 
dichotomous search (paired- 
trials), 460 
Fibonacci search, 455, 462-465 
Golden section, 455, 460-462 
lattice search, 455 
Search plan, 457-466 
Secular equation, 138 
Seek methods, 455 
(See also Search methods) 
Series, transcendental functions, 175 
Sets of equations: 
algebraic, 66 
first-order ordinary differential 
equations, 252, 254, 275 
higher-order ordinary differential 
equations, 259, 278 
nonlinear, 180 
Shift operator, 198 
Simpson’s one-third rule for integra- 
tion, 231, 244 
Simpson’s three-eighths rule for inte- 
gration, 274 
Spring-connected system: 
natural frequency of vibration, 140 
two-mass, 53, 54 
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Spurious solutions, 287 
Square root (Banachiewicz) method, 
81-83 
Square roots, 23 
Stability, numerical, initial-value 
problems, 286, 381, 385 
Star points, uncqual leg, 337-343, 371 
Starting series, 251 
Stationary point, 430, 442 
Steady-state problems, 1 
Steepest descent: 
method of, 467—478 
table of equations, 474 
Stencils, 324 
(See also Computational molecules) 
Subdivision method, 257 
Subscripted numbers: 
multidimensional arrays, 20 
one-dimensional array, 14 
two-dimensional arrays, 14 
Subsidiary equations, 434 
(See also Constraining equations) 
Summation: 
double, 11 
products, 11 
Synthetic division, 166-168 
Systems: 
continuous, 2 
deterministic, 433 
discrete element, 1 
ill-conditioned, 68 
lumped, 2 
nonlinear, 416 
static, 433 
stochastic, 4383 


Tangents, method of, 170, 171 
Taylor series solutions, ordinary dif- 
ferential equations, 247-252, 259, 
268 
Transcendental equations, real roots, 
175 
Transformation, coordinate, 42, 59 
Transpose: 
determinant, 27 
matrices, 45 
Trapezoidal integration rule, 230 
Truncation errors, 3, 217, 221, 287 
Truss, pin-connected: three-bar plane, 
473 
three-bar space, 56 
two-bar plane, 434 


Unimodality, 429, 456 

Unit matrix, 44 

Unit relaxation tables, 360 
Upper triangular matrix, 44 
Upper unit triangular matrix, 44 


Variation, calculus of, 488-455 
Vector: 

column, 41 

row, 41 


Wave propagation, 413 


Zero matrices, 44 
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